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Preface 


There  is  a  steadily  increasing  tendency  to  use  piloted  flight  simulators  for  official  clearance  of  selected  areas  of  flight  envelopes 
and  of  system  behaviour  or  malfunctions.  This  is  a  natural  and  desirable  evolution  from  the  wide  use  of  simulation  during  the 
development  of  new  aircraft.  However  there  is  a  lack  of  guidance  for  certification  authorities  and  aircraft  manufacturers  on 
simulation  standards,  validation  procedures  and  general  information  on  the  advantages  and  disadvantages  of  using  simulation  as 
part  of  a  clearance  programme.  This  could  lead  to  either  inappropriate  use  of  simulators,  or  unnecessary  (and  costly)  reluctance 
to  use  simulation  when  it  is  appropriate.  In  particular,  there  is  concern  by  many  involved  with  research  and  engineering 
development  simulators  that  subjective  pilot  opinion  is  often  the  primary  criterion  for  acceptance  of  simulators  for  certification 
activities.  Training  simulators  are  frequently  ‘adjusted’  to  improve  pilot  acceptance  for  that  role,  and  this  can  be  justified  by 
direct  comparison  with  the  aircraft.  However,  clearance  demonstrations  on  a  simulator  will  not  usually  be  experienced  in  flight 
until  an  operational  pilot  encounters  the  conditions  or  configurations  of  the  clearance.  Thus  validation  of  the  simulator  for 
clearance  tasks  must  involve  rigorous  model  and  simulation  system  validation  as  well  as  pilot  subjective  tests.  Subjective 
adjustments  are  unacceptable. 

AGARD  is  able  to  bring  together  experts  from  NATO  countries  into  Working  Groups  to  address  issues  of  common  interest. 
Working  Group  16  was  formed  by  the  Flight  Mechanics  Panel  at  the  end  of  1987  to  produce  this  Advisory  Report.  The  aim  was 
to  provide  advice  and  guidance  to  Certification  and  Acceptance  Authorities,  and  Aircraft  Manufacturers  on  the  appropriate  use  of 
piloted  simulation  as  the  sole  demonstration  for  aircraft  and  system  flight  clearances. 

The  Group  included  members  from  Canada,  Germany,  Italy,  Netherlands,  United  Kingdom  and  the  United  States.  Government 
R&D  establishments,  Armed  Service  R&D  establishments  and  aircraft  and  simulator  manufacturers  were  all  represented. 
Contributions  to  this  report  were  provided  by  all  members  of  the  Working  Group,  who  also  provided  valuable  assistance  to  the 
Chairman  in  the  editing  process. 


Alan  A  Woodfield 
Chairman  FMP  WG-16 
Member,  Flight  Mechanics  Panel 
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Preface 


II  existe  une  tendance,  de  plus  en  plus  marquee,  d’utiliser  les  simulateurs  de  vol  pilotes  pour  1’ homologation  de  certaines  parties 
du  domaine  de  vol  et  du  comportement  des  systemes  en  cas  de  mauvais  fonctionnement.  En  fait,  il  s’agit  d’une  evolution 
naturelle  et  souhai table  vers  1’ utilisation  generalisee  de  la  simulation  aux  fins  du  developpement  des  nouveaux  aeionefs. 

Cependant,  les  services  officiels  charges  de  la  certification  et  les  avionneurs  ne  disposent  pas  de  suffisamment  de  consignes  sur 
les  normes  de  simulation,  les  procedures  de  validation  et  les  avantages  et  desavantages  en  general  de  I’emploi  de  la  simulation 
dans  le  cadre  d’un  programme  d ’homologation.  Cette  situation  pourrait  conduire  soit  a  une  utilisation  mappropriee  des 
simulateurs,  soit  a  une  certaine  resistance  inutile  (et  couteuse)  a  l’emploi  de  la  simulation  la,  ou  elle  est  necessaiie.  En  particular, 
beaucoup  de  ceux  qui  sont  impliques  dans  le  developpement  des  simulateurs  de  conception  et  de  recherche  se  disent  preoccupes 
par  le  fait  que  I’avis  subjectif  du  pilote  est  souvent  le  critere  d’acceptation  principal  d’un  simulateur  dans  le  cadie  de  la 
certification. 

Les  simulateurs  d’entramement  sont  souvent  “ajustes”  afin  de  faciliter  leur  acceptation  dans  cette  mission  par  le  pilote,  ce  qui 
s’avere  justifiee  lorsque  la  comparaison  est  faite  avec  un  avion  en  vol.  Cependant,  les  demonstrations  pour  certifications  sur 
simulateur  ne  sont  normalement  verifiees  par  le  pilote  operationnel  qu’au  moment  ou  il  rencontre  en  vol  les  conditions  ou  les 
configurations  de  cette  rneme  certification.  Il  s’ensuit  que  la  validation  des  simulateurs  pour  des  taches  de  simulation  passe 
necessairement  par  la  certification  rigoureuse  des  systemes  de  moderation  et  de  simulation,  ainsi  que  par  les  tests  subjectifs  des 
pilotes.  Les  ajustements  subjectifs  sont  inacceptables. 

L’AGARD  a  la  capacite  de  reunir  des  specialistes  des  pays  membres  de  l’OTAN  dans  le  cadre  d’un  groupe  de  travail  porn 
examiner  des  questions  d’interet  mutuel.  Le  groupe  de  travail  No.  16  a  ete  cree  par  le  Panel  de  la  Mecanique  du  Vol  a  la  fin  de 
l’annee  1987  avec  pour  mandat  l’elaboration  du  present  rapport  consultatif.  Le  groupe  s’est  donne  comme  objectif  de  foumir  des 
conseils  et  de  formuler  des  orientations  concemant  [’utilisation  appropriee  de  la  simulation  pilotee  en  tant  que  moyen  unique  de 
demonstration  en  vue  de  l’homologation  des  aeronefs  tant  du  point  de  vue  de  leurs  performances  de  vol  que  de  leurs  systemes, 
aux  autorites  responsables  de  la  certification  et  de  la  reception  des  aeronefs,  ainsi  qu’aux  fabricants  des  simulateurs  et  aux 
avionneurs. 

Le  groupe  a  ete  compose  de  representants  du  Canada,  de  l’Allemagne,  de  l’ltalie,  des  Pays  Bas,  du  Royaume-Uni  et  des  Etats- 
Unis  appartenant  a  des  etablissements  de  recherche  et  developpement  gouvemementaux  et  militaires,  ainsi  que  de  lepiesentants 
d’avionneurs  et  de  fabricants  de  simulateurs. 

Des  contributions  au  present  rapport  ont  ete  foumies  par  tous  les  membres  du  groupe,  qui  ont  en  outre  bien  voulu  preter  leui 
concours  precieux  au  president  lots  de  sa  redaction.  Des  contributions  a  ce  rapport  ont  ete  foumies  par  l’ensemble  des  membres 
du  Groupe  de  Travail,  qui  ont  egalement  prete  de  l’aide  au  President  lors  de  la  phase  de  relectuie  du  document  definitif. 

Alan  A  Woodfield 

President  FMP  WG-16 

Membre,  Panel  de  la  Mecanique  du  Vol 
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1  INTRODUCTION 

Piloted  flight  simulators  are  a  powerful  tool  for 
designers  to  study  the  influence  of  the  pilot  in  the 
dynamic  behaviour  of  new  aircraft,  new  systems  and 
new  operating  environments.  They  also  provide  training 
for  pilots  in  managing  the  complex  cockpit  environment 
and  the  wide  range  of  emergency  conditions  which 
might  arise  from  failures  or  adverse  operational  condi¬ 
tions.  For  many  years  simulators  have  been  used 
successfully  to  predict  and  assess  solutions  to  problems 
arising  during  the  development  of  new  aircraft  and 
systems.  However,  until  recently,  with  the  exception  of 
very  specialised  aircraft  such  as  the  US  Space  Shuttle, 
all  aircraft  acceptance/certification  has  required  flight 
test  demonstration. 

This  situation  is  changing  as  the  complexity  of  safety 
critical  systems  increases  and  presents  the  acceptance 
authority  with  a  very  large  set  of  potential  failure 
modes.  Also,  improvements  in  the  standards  of  flight 
simulation  are  increasing  confidence  in  their  ability  to 
represent  many  flight  situations  well  enough  to  limit  the 
range  of  conditions  that  require  flight  testing.  By 
holding  flight  clearance  testing  within  reasonable 
bounds  there  are  significant  savings  in  cost  and  time, 
which  are  eagerly  embraced  by  both  manufacturer  and 
customer.  There  are  now  examples  where  piloted  simu¬ 
lation  has  been  used  to  'demonstrate'  to  acceptance 
authorities  a  range  of  failure  modes  of,  for  example,  a 
multi-channel  'fly-by-wire'  flight  control  system.  From 
these  demonstrations,  the  authorities,  quite  reasonably, 
selected  for  flight  demonstration  those  situations  that 
appeared  to  be  most  demanding  and  most  probable. 
Economic  and  practical  time  limitations  are  going  to 
increase  the  range  of  situations  where  piloted  simulation 
will  be  used  as  a  direct  part  of  the  acceptance  and  certi¬ 
fication  processes  for  military  and  civil  aircraft  and  their 
systems. 

This  increasing  use  of  piloted  simulation  for  acceptance 
testing  of  aircraft  is  a  natural  consequence  of  the 
increasing  physical  understanding  of  aircraft  and  their 
operational  environment,  and  confidence  in  the  'validity' 
of  piloted  simulation.  Confidence  in  validity  is  the 
criterion  that  must  be  applied.  How  is  such  confidence 
derived? 

In  the  above  case  of  multi-channel  'fly-by-wire'  systems, 
confidence  was  obtained  primarily  from  the  subjective 
comments  of  experienced  pilots  involved  in  develop¬ 
ment  flight  testing  of  the  aircraft.  This  is  an  important 
and  necessary  element  of  validation.  Is  it  sufficient?  The 
wide  experience  of  members  of  the  Working  Group  is 
that  pilots'  subjective  comments  are  not  sufficient  on 
their  own  to  justify  confidence  in  the  validity  of  a 
simulation.  For  example,  it  is  not  unusual  in  developing 
training  simulators  to  try  to  compensate  for  physical 
cueing  deficiencies,  or  computing  delays,  by  altering  the 
aircraft  model  to  make  the  simulator  appear  to  be  more 
"like  the  aircraft"  than  the  earlier  version  with  the  more 


representative  aircraft  model.  This  method  of  compen¬ 
sating  for  cueing  deficiencies  is  possibly  acceptable  for 
a  training  simulator,  where  the  responses  can  be  tuned 
to  be  a  satisfactory  representation  of  specific  training 
tasks  that  are  demonstrated  on  the  aircraft.  (Although 
even  with  training  simulation  there  can  be  problems 
when  users  subsequently  modify  the  training  syllabus  to 
use  a  wider  operational  envelope.) 

For  acceptance  test  purposes,  however,  it  is  not  accept¬ 
able  to  alter  aircraft/system  models  to  compensate  for 
cueing  deficiencies.  Although  this  can  gain  pilot  accep¬ 
tance  and  confidence  in  the  areas  which  he  has  experi¬ 
enced  in  flight,  there  can  be  little  confidence  that  the 
simulator  is  presenting  adequately  those  vital  situations 
which  will  not  be  demonstrated  in  flight.  Further,  the 
use  of  such  model  compensation  can  be  highly  mislead¬ 
ing  because  pilots  and  engineers  from  both  the  manufac¬ 
turer  and  the  acceptance  authorities  can  be  led  to  a  false 
sense  of  confidence,  and  the  simulation  engineer  can 
believe  that  the  simulation  is  valid  in  general.  All 
parties  need  to  ask  the  questions: 

a  For  what  sets  of  conditions  has  validity  been 
confirmed  from  flight  test? 

b  What  confidence  can  be  placed  in  simulation  of 
conditions  outside  the  validated  range? 

The  ail  of  optimising  a  simulation  to  maximise  its  valid¬ 
ity  for  specific  tasks  depends  on  a  w  ide  range  of  factors, 
including  the  physical  characteristics  of  available 
cueing  systems  (visuals,  motion,  sound,  control  loading, 
g-seats,  etc.),  computing  systems  (architecture,  speed, 
capacity,  etc.),  and  aircraft  category.  Most  cueing 
systems,  with  the  important  exceptions  of  visuals  and 
motion,  have  only  small  delays.  Visual  scene  generation 
often  takes  several  computing  cycles  and  the  drive 
algorithms  may  contain  rate  and  acceleration  terms  to 
try  and  compensate  for  the  delays,  which  are  typically 
between  70  and  140  milliseconds.  Such  compensation 
must  be  used  with  care  as  it  is  easy  to  overcompensate 
and  introduce  unnatural  overshooting  of  the  correct 
visual  scene  following  energetic  manoeuvres.  Motion 
drive  algorithms  are  complex  and  contain  both  scaling 
and  dynamic  response  filters  designed  to  generate 
representative  aircraft  movement  cues  over  a  limited 
bandwidth  of  frequencies  that  are  compatible  with  the 
acceleration,  velocity  and  movement  limits  of  the 
motion  system.  Thus  acceptance  authorities  and  manu¬ 
facturers  need  to  justify  confidence  in  simulation  results 
from  either  relevant  flight  test  validation  or  from  an 
identification  and  acceptance  of  the  validity  of  the 
simulation  system  design  methods  applied  to  the 
specific  simulator.  Pilot  acceptance  is  not  sufficient, 
although  it  will  always  be  one  of  the  necessary  condi¬ 
tions. 

This  Advisory  Report  collects  the  views  and  experience 
of  international  experts  in  flight  simulation,  and  seeks  to 
inform  manufacturers  and  acceptance  authorities  of  past 
experience  with  the  use  of  simulation  in  both 
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development  and  acceptance/certification  testing.  It 
discusses  a  wide  range  of  practical  issues  arising  in 
modelling,  providing  cues,  and  integrating  aircraft  and 
simulation  systems.  It  introduces  guidance  on  validation 
methods,  outlines  issues  which  may  need  to  be  consid¬ 
ered  by  acceptance  authorities,  and  considers  areas 
where  simulation  is  likely  to  be  a  major  contributor  in 
acceptance  testing.  The  main  body  of  the  report  is 
devoted  to  simulator  validation:  the  factors  involved, 
and  the  techniques  which  are  available.  In  the 
Appendices,  specific  examples  are  given  of  the  use  of 
flight  simulation  for  certification  and  customer  accep¬ 
tance  of  both  military  and  civil  aircraft.  These,  and 
more  recent  examples,  all  confirm  the  general  conclu¬ 
sions  of  this  report. 

2  CURRENT  USE  OF  SIMULATION  FOR 
CERTIFICATION/ACCEPTANCE 

2.1  Civil  Certification  Experience 

The  use  of  piloted  simulation  by  the  aircraft  manufac¬ 
turer  in  the  design  and  development  of  new  civil  trans¬ 
port  aircraft  is  very  extensive.  No  aircraft  manufacturer 
today  would  undertake  the  launching  of  a  new  aircraft 
programme  without  major  usage  of  simulation.  First, 
sophisticated  computer  models  will  be  used  in  a  proof  of 
concept,  to  predict  the  aeroplane's  stability  margins, 
estimate  performance  characteristics  and  analyse  hand¬ 
ling  qualities  and  failure  mode  effects.  Second,  piloted 
simulations  will  be  developed  from  these  models  and 
used  to  evaluate  manoeuvres,  procedures  and  systems 
before  they  are  cleared  on  the  aircraft  and  to  provide  a 
confidence  factor  for  the  aeroplane's  certification.  The 
areas  of  application  range  from  pure  system  develop¬ 
ment,  with  and  without  prototype  hardware  in  the  loop, 
through  crew  workload  studies  in  normal,  abnormal  and 
emergency  procedures,  to  the  assessment  of  handling 
qualities  in  all  flight  phases  for  specific  control  law 
design. 

However,  compared  to  the  extensive  use  of  simulation 
in  aircraft  design,  the  direct  application  of  simulation  in 
certification  tests  is  currently  still  limited  to  the  assess¬ 
ment  of  (sub)systems  failure  cases  that  are  very  difficult 
to  obtain  in-flight.  The  simulator  has  proved  to  be 
particularly  suited  for  that  purpose  because  it  provides 
time  to  analyse  the  warnings  and  procedures,  to  repro¬ 
duce  the  tests  as  many  times  as  is  required,  and  to  cover 
a  wider  spectrum  of  flight  conditions.  In  addition  it 
avoids  exposing  people's  lives  and  aircraft  to  conditions 
of  potentially  high  risks,  and  it  is  possible  to  create 
failure  cases  in  controlled  meteorological  conditions 
that  are  otherwise  difficult  or  impossible  to  obtain  to 
order  from  the  natural  environment  (e.g.  turbulence, 
wind  shear,  etc.).  Finally,  it  helps  the  manufacturer  to 
define  the  certification  flight  programme  proposed  to 
the  certifying  authorities. 

No  specific  rules  exist  by  which  authorities  accept  that 
proof  of  compliance  with  a  certain  requirement  can  be 
given  by  simulation.  Further,  Airworthiness 


Requirements1  do  not  dictate  the  acceptable  means  of 
compliance  (analysis,  ground  or  flight  test  with  the 
actual  aircraft  or  flight  simulation)  and  leave  the  choice 
to  the  manufacturer.  The  aircraft  manufacturer  always 
takes  the  lead  in  proposing  the  means  for  proof  of 
compliance  with  a  certain  certification  requirement.  The 
decision  to  propose  simulation  or  flight  test  is  governed 
by  economics,  taking  into  account  all  factors  that  are 
expected  to  be  of  influence.  These  may  be  expense, 
available  time,  risks  involved,  availability  of  equipment 
and  human  resources,  probability  of  success,  anticipated 
attitude  of  airworthiness  authorities  etc.  The  final 
division  between  actual  flight  testing  and  alternative 
means  is  the  result  of  negotiations  between  the  authority 
and  the  manufacturer  and.  amongst  other  factors,  is 
dependent  on  the  availability  of  "proper"  flight  testing 
and  simulation  facilities. 

Current  practice  is  for  compliance  with  airworthiness 
regulations  in  normal  aircraft  operation  to  be  demon¬ 
strated  in  flight  test.  Failure  cases  with  major  conse¬ 
quences  and  w  ith  a  probability  <  10'7  will  require  either 
flight  or  simulator  tests.  The  type  of  test  is  at  the  discre¬ 
tion  of  the  airworthiness  authorities.  Other  failures  with 
hazardous  consequences  or  with  a  probability  >10'7  are 
assessed  in  simulator  tests.  Performance  of  most  avionic 
systems  and  failure  cases  are  usually  certificated  in  the 
simulator  with  some  flight  tests  to  validate  the  simulator 
results. 

However,  if  the  Authorities  suspect  that  the  effect  of  a 
failure  on  flight  crew'  workload  and/or  ease  of  informa¬ 
tion  interpretation  is  influenced  by  the  validity  of  the 
simulation,  they  will  always  require  in-flight  verifica¬ 
tion  regardless  of  the  results  of  the  simulator  tests.  This 
happens  for  instance  when  a  new  aircraft  has  very  novel 
and  unusual  features,  in  which  case  flight  tests  may  be 
required  even  for  very  rare  failures. 

An  important  function  of  simulation  regarding  airwor¬ 
thiness  certification  is  that  before  or  even  without 
actually  flying,  the  certification  pilots  can  get  confident 
with  the  control  and  display  systems  and  study  specific 
flight  control  and  instrumentation  design  issues  in  a 
wide  spectrum  of  flight  conditions,  before  being 
subjected  to  the  additional  workload  of  a  real  flight. 
When  the  complexity  of  a  system  increases  signifi¬ 
cantly,  as  with  integrated  EFIS,  automated  flight  control 
and  central  warning  systems,  the  need  to  have  the  total 
picture  in  a  real  aircraft  environment  during  tests 
increases  accordingly.  The  simulator  is  then  very 
helpful  in  evaluating  the  system  features  and  pin¬ 
pointing  the  bottle-necks  as  a  preparation  for  the  actual 
flight  tests  (e.g.  to  demonstrate  effects  of  new  software 
versions  of  certain  systems). 

In  most  cases  the  simulation  facility  offered  will  have 
some  limitations  compared  to  the  aircraft  or  prototype 
because  the  aerodynamic  data  are  not  yet  fully  known, 
or  because  the  simulator  has  limitations  in  vision, 
motion  or  control  capabilities,  or  because  actual  flight 
hardw'are  is  not  available.  As  for  training  simulators,  the 
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manufacturers  have,  on  request,  to  provide  the  certify¬ 
ing  authorities  with  objective  information  concerning 
the  validity  of  the  simulation  for  the  proposed  task.  A 
final  decision  to  accept  that  the  simulation  is  suitable 
for  the  proposed  task  is  made  only  after  a  subjective 
simulation  assessment  by  a  certification  pilot.  An 
important  aspect  in  this  decision  process  is  whether  the 
quality  of  the  simulation  is  such  that  the  simulation 
results  can  be  considered  as  conservative.  If  the  diffi¬ 
culty  of  the  piloting  task  is  increased  slightly  in  simula¬ 
tion  with  respect  to  real  flight  this  would  normally  be 
more  acceptable  than  a  corresponding  reduction  in 
difficulty.  However,  it  is  no  longer  safe  to  assume,  as 
was  frequently  true  in  the  past,  that  flying  a  simulator 
will  be  more  difficult  than  the  aircraft.  The  introduction 
of  active  flight  control  systems  has  produced  several 
instances  where  a  simulator  has  been  easier  to  fly  than 
the  aircraft.  Experience  shows  that,  when  in  the  course 
of  a  simulation  programme  the  confidence  of  the  certifi¬ 
cation  team  in  the  simulation  facilities  and  the  test 
proposals  of  the  manufacturer  increases,  the  amount  of 
in-flight  validation  required  will  decrease.  Since 
currently  this  varies  from  country  to  country,  and  within 
a  country  from  aircraft  to  aircraft,  it  may  be  that  it  will 
be  difficult  to  draw  up  a  general  set  of  rules  for  accept¬ 
ing  that  simulation  is  applicable.  However,  with  the 
advent  of  a  Joint  Airworthiness  Authority  in  Europe, 
with  teams  of  experts  handling  the  same  part  of  the 
certification  process  for  all  aircraft  offered  for  certifica¬ 
tion,  it  may  very  well  be  possible  and  even  necessary 
that  general  rules  will  emerge  in  the  future.  Guidance  on 
crucial  issues  will  be  necessary'  until  that  time. 

Valuable  experience  in  civil  airworthiness  certification 
was  gained  during  the  certification  programmes  of  the 
Fokker  50  and  Fokker  100.  and  in  the  joint  European 
Airworthiness  certification  programmes  of  the  Airbus 
A-320  and  the  Boeing  747-400  aircraft.  For  a  significant 
number  of  aircraft  sub-systems,  the  authorities  have 
already  accepted  that  flight  simulation  programmes  can 
contribute  in  providing  proof  of  compliance  with  air¬ 
worthiness  requirements.  Appendix  A  presents  a  number 
of  specific  examples. 

During  the  A3  20  development  programme  (Appendix 
A),  the  simulator  played  a  very  significant  role  in 
predicting  accurately  the  response  of  the  aircraft  under 
various  testing  configurations.  However,  two  specific 
flight  handling  characteristics  were  found  to  be  lacking 
in  realism  on  the  simulator. 

i.  In  studying  the  behaviour  of  the  aircraft  at  aft 
CG  limits  in  the  "direct  control  law"  mode,  the 
simulator  was  found  to  be  more  demanding  to 
fly  than  the  aircraft.  This  was  attributed  to  the 
following  factors: 

a)  The  simulator  had  no  motion  system. 

b)  At  time  of  test,  the  latest  aerodynamic  data 
were  not  used.  It  was  concluded  that  small 
differences  in  longitudinal  stability  mar¬ 


gins  at  aft  CG  can  cause  significant  vari¬ 
ations  in  handling  qualities.  (Even  after 
updating  the  aerodynamic  model,  it  was 
found  that  pilot  inputs  in  the  simulator 
tended  to  be  larger  than  those  used  in 
flight,  and  the  simulator  handling  task  was 
more  demanding  than  on  the  aircraft. 
However,  the  simulator  response  character¬ 
istics  with  the  system  stabilisation  in  the 
augmented  mode,  was  representative  of  the 
aircraft.) 

ii.  Some  early  versions  of  the  control  laws  were 
evaluated  as  satisfactory  on  the  simulator 
during  the  landing  and  flare  phases  of  flight. 
However,  these  were  found  to  be  unacceptable 
on  the  aircraft  during  flight  in  significant  gust 
or  cross-wind  conditions.  This  was  attributed  to 
the  lower  level  of  pilot  stress  experienced  on 
the  simulator  and  the  absence  of  a  motion 
system.  The  real  aircraft  had  to  be  used  exten¬ 
sively  to  refine  the  longitudinal  control  laws  to 
obtain  the  desired  flare  characteristics  during 
various  weather  conditions. 

2.2  Military  Acceptance  Experience 
Flight  simulation  is  more  widely  used  to  support  the 
acceptance  of  military'  aircraft  than  in  the  certification 
of  civil  aircraft.  The  flexibility  of  operation  which  char¬ 
acterises  military  aircraft  usage  means  that  the  flight 
testing  required,  both  for  aircraft  development  and  for 
aircraft  clearance,  is  far  more  extensive  than  that 
required  for  the  equivalent  clearances  on  civil  aircraft. 
As  well  as  operating  over  a  much  wider  flight  envelope, 
the  configuration  of  military'  aircraft  vary  with  the 
carriage  of  external  stores,  adding  to  the  number  of 
cases  which  must  be  flown. 

New  military  aircraft  are  designed  and  constructed  as  a 
result  of  protracted  discussions  between  Industry, 
Government  Departments,  Armed  Forces,  and  Agencies 
responsible  for  procurement.  Although  export  appeal 
will  have  some  influence  on  the  decision  to  proceed, 
National  requirements  prevail.  In  the  case  of  a  new  civil 
aircraft,  commercial  pressures  dominate,  and  the  manu¬ 
facturer  will  design  an  aircraft  with  world-wide  appeal 
and  with  an  eye  to  International  Certification. 
Consequently,  the  procedures  for  design,  test,  and 
acceptance  of  military'  aircraft  show  a  much  greater 
variation  between  countries  than  those  relating  to  civil 
aircraft.  In  the  United  States  the  procedures  are  con¬ 
tained  in  Mil  Standards,  in  the  UK  in  Defence  Standard 
00-970,  and  so  on.  For  multi-national  programmes,  the 
procuring  authority  which  represents  the  participating 
nations  will  formulate  new  requirements  based  on  indi¬ 
vidual  national  requirements.  Consequently,  the  basis 
for  military  aircraft  acceptance  does  not  share  common 
ground  to  anything  like  the  extent  to  be  seen  in  the 
certification  of  civil  aircraft. 

At  the  same  time,  the  various  national  requirements  for 
military  aircraft  acknowledge,  to  a  greater  extent  than 
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die  civil  requirements,  die  need  for  die  use  of  simulation 
to  support  die  acceptance  process.  Section  5.2  expands 
on  this  theme.  It  is  a  reflection  of  die  extensive,  and 
sometimes  hazardous,  nature  of  the  test  flying  which  is 
involved.  The  aircraft  must  be  cleared  to  extreme  points 
of  the  flight  envelope  in  a  wide  range  of  loading  condi¬ 
tions.  weights,  and  store  configurations.  Rapid  rolling 
and  spinning  tests  require  particular  care.  Testing 
special  equipment,  such  as  radar,  stores  management, 
and  weapon  delivery  systems,  are  demanding  on  flight 
time,  and  call  for  facilities  such  as  low-flying  areas, 
ranges,  and  ground  and  air  targets. 

The  common  factor  between  military  and  civil  practice 
in  the  use  of  simulation  for  acceptance  and  certification 
is  that  die  agreement  to  do  such  testing  has  to  be 
approved  by  the  certifying  authority.  Each  case  is 
discussed  on  its  merits,  depending  on  precedent,  the 
available  standard  of  simulator,  and  goodwill  on  both 
sides.  The  benefits  are  widely  recognised,  including  the 
opportunities  diat  die  simulators  give  for  pilot  familiari¬ 
sation  and  their  ability  to  represent  conditions  such  as 
turbulence  which  are  rarely  available  exactly  when 
required  in  flight. 

Appendix  B  contains  a  number  of  examples  of  flight 
simulation  activities  w  hich  have  helped  in  the  clearance 
of  recent  military  aircraft.  They  illustrate  die  wide 
diversity  of  problems  which  have  been  cleared  by 
simulation,  on  many  types  of  aircraft.  They  cover  a 
wide  range  of  specialities  such  as  flying  qualities,  flight 
control,  avionics,  stalling  and  spinning,  ground 
handling,  terrain  following,  weapon  aiming  and  failures. 
The  examples  of  Appendix  B  are  just  those  instances  of 
clearance  work  known  to  members  of  the  Working 
Group.  Most  military  aircraft  entering  service  in  the  past 
twenty  years  have  benefited  from  simulator  tests  in 
support  of  flight  clearance.  The  advent  of  new  types  of 
aircraft,  such  as  the  X-31  and  the  forward  swept  wing 
X-29  before  it.  will  increase  the  need  to  predict  aircraft 
and  flight  control  performance  prior  to  first  flight. 
Simulators  can  take  some  of  the  risk  out  of  flight  tests  if 
die  laws  of  motion  and  aerodynamics  are  accurately 
applied.  The  use  of  simulators  to  support  aircraft  devel¬ 
opment  and  certification  will  inevitably  increase  with 
the  increasing  development  costs  of  military1  aircraft  and 
their  correspondingly  higher  cost  per  flight  hour. 

3  VALIDATION  ISSUES 

This  section  "sets  the  scene"  for  the  process  of  valida¬ 
tion.  The  implications  of  decisions  taken  during  devel¬ 
opment  of  the  simulation  are  discussed,  and  a  number  of 
potential  problem  areas  are  identified.  In  many 
instances,  the  issues  which  arise  have  a  bearing  not  just 
on  validation,  but  indeed  on  the  very  feasibility  and 
practicality  of  the  use  of  simulation  in  the  certifica¬ 
tion/acceptance  process.  It  is  convenient  to  address 
these  aspects  briefly  here.  Advice  on  specific  methods 
of  carrying  out  validation  is  covered  in  Section  4. 


Before  proceeding  further  it  is  important  to  provide  a 
definition  of  validity  in  this  context  of  piloted  simula¬ 
tion.  The  definition  used  here  is: 

A  piloted  simulation  will  be  valid  for  a  selected 
task  when  the  representation  by  individual 
components  of  the  simulation  and  by  the 
complete  simulation  meets  both  the  quantita¬ 
tive  requirements  of  the  manufacturer/authority 
and  the  subjective  evaluation  of  pilots  with 
relevant  experience  such  that  a  manufacturer 
will  confidently  present  simulation  as  the  sole 
means  of  demonstrating  compliance  with  the 
Authority's  certification  or  acceptance  require¬ 
ments  for  some  element  of  that  task. 

Before  proceeding  it  is  also  appropriate  to  define  the 
term  'verification'.  This  is  used  here  for  the  more 
specific  action  of  objectively  comparing  the  response  of 
a  component  of  simulation  w'ith  its  specification,  e.g.  is 
the  implementation  of  a  mathematical  model  a  correct 
representation  of  the  specified  model.  Verification  is 
usually  a  precise  objective  comparison,  whereas  valida¬ 
tion  contains  both  verification  and  subjective  assess¬ 
ments  to  evaluate  the  fitness  of  the  system  for  the 
required  purpose. 

Validation  covers  all  aspects  of  the  complete  simulation 
as  shown  in  Fig  1  -  the  mathematical  model  of  the 
vehicle,  methods  used  to  represent  on-board  systems 
and  equipment,  the  atmospheric  and  operational 
environment  to  be  represented,  the  software  needed  to 
create  all  these  components,  and.  most  important  in  this 
context,  the  ways  in  which  these  systems  interface  with 
a  the  pilot.  Issues  relating  to  the  interactions  between 
pilots  and  flight  simulators  are  probably  the  most 
complex  and  contentious  and  these  are  considered  first. 
Each  of  tire  other  aspects  is  then  considered  in  turn  in 
the  sub-sections  below. 

3.1  Pilot  Interaction 

Piloted  simulation  wall  be  appropriate  for  situations  and 
configurations  where  acceptance  depends  on 

a  pilot's  dynamic  response  behaviour  (handling 
qualities,  w  orkload  and  control  activity) 

b  piloting  procedures,  particularly  under  high 

stress  (trained  procedures) 

c  complex  situation  monitoring  near  the 

boundaries  of  safe  flight  (safety  margins). 

(These  are  exactly  the  same  requirements  that  apply  for 
selecting  flight  acceptance  tests  as  the  involvement  of 
aircrew  is  a  common  feature  to  both  flight  test  and 
piloted  simulation.) 


The  validity  of  results  from  a  simulation  depend  directly 
on  its  capabilities  to  stimulate  the  same  task  perform¬ 
ance.  subjective  opinion  of  handling  qualities  and 
workload,  and  control  activity  from  a  pilot  as  would  be 
found  in  the  aircraft  for  the  chosen  tasks. 
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Fig  1:  Piloted  Simulation  System 


It  is  important  to  put  'validity'  and  ’realism'  into  proper 
perspective  as  they  are  not  equivalent.  Although  striving 
for  'realism'  will  improve  the  general  quality  of  a  simu¬ 
lation,  it  is  in  the  end  an  unachievable  goal.  Simulators 
will  always  differ  from  flight  in  various  ways  and  to 
greater  or  lesser  degrees.  One  example  is  the  simple 
flight  manoeuvre  of  changing  from  straight  flight  to  a 
steady  turn.  No  ground  based  simulator  can  ever 
produce  correctly  both  the  acceleration  transients  when 
setting  up  the  turn  and  the  steady  'g'  load  in  the  turn. 
Degrees  of  realism  are  ill-defined  and  increased  realism 
will  only  enhance  validity  if  it  is  applied  to  simulation 
features  which  directly  affect  the  chosen  tasks,  e.g. 
improving  the  detail  of  airport  buildings  will  not 
improve  the  validity  of  a  simulator  in  its  response  to 
wind  shear.  Validity  depends  on  establishing  a  high 
level  of  confidence  that  pilot  responses  and  perform¬ 
ance  in  the  simulator  will  be  the  same  as  in  the 
aircraft  for  the  chosen  tasks. 

This  section  addresses  issues  that  affect  pilot  responses 
and  for  convenience  separates  these  into  handling  and 
situation  awareness  activities.  A  summary  of  pilots 
dynamic  behaviour  is  followed  by  discussions  on 
dynamic  cueing  systems.  Situation  cueing  systems  are 
followed  by  a  sub-section  on  pilot  adaptability',  stress 
and  factors  affecting  the  pilot's  subjective  assessment  of 
the  simulator.  Always  remembering  that,  although  pilot 
subjective  assessment  of  validity  is  not  sufficient  on  its 
own,  no  simulator  will  be  accepted  as  valid  unless  pilot 
opinion  is  favourable. 


3.1.1  Pilot  dynamic  sensing  and  response 
People  have  a  wide  variety  of  sensors  with  which  they 
identify  what  is  happening  to  them  and  their  immediate 
world.  These  sensors  often  provide  signal  redundancy 
and  processing  of  signals  is  a  complex  activity  including 
learned  responses.  Some  of  the  apparent  inconsistencies 
in  pilot  assessments  of  simulation  arise  from  their 
adaptability  (ability  to  learn).  Others  are  often  associ¬ 
ated  with  the  fact  that  some  sensors  contribute  to  an 
overall  sense  of  situation  without  giving  a  precise 
quantitative  measurement,  e.g.  motion  sensors  of  the 
inner  ear  are  imprecise  on  their  own  but  provide  good 
control  when  combined  with  limb  position  sensing  and 
even  better  control  when  vision  is  added.  This  does  not 
mean  that  the  inner  ear  sensors  are  not  vital,  as  is  clear 
from  the  distress  and  difficulties  in  balancing  caused 
when  they  are  damaged,  but  it  does  mean  that  pilots 
may  not  be  aware  whether  a  particular  motion  cue  is  or 
is  not  present.  They  will,  however,  usually  identify 
changes  in  the  behaviour  of  the  vehicle  as  all  motion  is 
turned  on  or  off.  However,  where  motion  cues  are 
important  for  a  task,  there  will  be  effects  on  some  if  not 
all  of  the  key  measures  of  validity,  i.e.  task  perform¬ 
ance,  subjective  opinion  of  handling  qualities  and 
workload,  and  control  activity.  If  task  performance  or 
subjective  opinion  are  affected  then  it  is  not  too  difficult 
to  assess  the  effects  on  validity.  If  only  control  activity 
is  changed  then  pilots  may  very  well  have  adapted  to 
the  behaviour  of  the  simulator  without  noticing  that  they 
are  using  different  control  inputs  from  those  that  would 
be  used  in  the  aircraft  in  the  same  situation. 
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The  whole  process  of  learning  to  respond  to  the 
complex  mixture  of  sensory  inputs  experienced  in 
flying,  or  even  ordinary  daily  activities,  is  only  partly 
understood.  People  are  able  to  adapt  quite  rapidly  to 
sensory  deprivation  of  many  different  kinds.  Quite  often 
task  performance  is  unaffected:  sometimes  they  will 
have  to  work  harder  to  achieve  this  performance, 
whereas  at  other  times  they  find  a  different  way  of 
achieving  the  same  result.  However,  this  different  way 
may  be  inappropriate  in  the  full  sensory  environment. 

The  stronger  senses  of  vision,  touch,  hearing  and 
smell/taste  will  dominate  any  appreciation  of  situation, 
but  confusion  may  arise  if  other  senses  do  not  agree 
with  the  stronger  senses.  These  effects  can  be  subtle,  as 
described  above,  or  more  obvious,  e.g.  the  difficulty 
experienced  in  maintaining  balance  when  viewing  large 
screen  action  film  w  ithout  any  accompanying  motion. 

In  considering  pilot  dynamic  responses  it  is  convenient 
to  consider  the  person  as  a  complex  servo  control 
system  including  both  sensors  and  actuators,  Fig.l,  in 
order  to  understand  the  main  consequences  of  differ¬ 
ences  between  motion  feedback  in  flight  and  on  a  par¬ 
ticular  simulator.  Any  closed  loop  servo  system,  e.g. 
where  rate  and/or  acceleration  feedback  is  used  to 
improve  the  control  of  position,  can  become  unstable  as 
the  gain,  i.e.  the  amount  of  input  to  adjust  for  a  given 
error  in  output,  is  increased,  or  as  the  time  lead  (phase 
lead)  of  the  feedback  decreases.  Real  acceleration  gives 
more  lead  than  velocity  (rate)  feedback,  which  in  turn 
leads  position  (attitude).  Thus  removing  the  acceleration 
cues  provided  by  motion  will  reduce  the  capability  of  a 
pilot  to  stabilise  and  control  the  simulated  aircraft. 

Direct  vision  of  the  outside  world  provides  position 
information.  Velocity  information  can  be  derived  by 
observing  successive  positions  but  this  lags  rather  than 
leads  position.  Direct  rate  information  is  obtained  in 
peripheral  vision  but  is  largely  direction  of  velocity 
rather  than  quantitative  information.  Sensors  in  tire  inner 
ear  and  muscles  respond  directly  and  progressively  to 
velocity  and  acceleration  and  these  provide  stabilising 
lead  in  flight.  In  a  simulator  there  is  no  natural  connec¬ 
tion  between  artificial  visual  scenes  and  the  behaviour 
of  motion  systems.  Indeed  the  limits  on  total  movement 
in  a  simulator  mean  that  it  is  impossible  to  represent  the 
full  relationships  between  visual  and  platform  motion. 
Fortunately,  human  sensing  and  response  frequencies 
are  limited  and  there  are  sensory'  thresholds.  These 
mean  that  important  and  relevant  stabilising  lead  can  be 
provided  within  the  limitations  of  practical  simulation 
motion  systems. 

The  influence  of  simulation  approximations  in  motion 
and  visual  on  a  pilot's  control  activity  compared  with 
the  same  situation  in  flight  will  depend  on  the  required 
precision  and  agility  of  the  task,  and  also  the  stability 
margin  at  normal  pilot  gains.  If  simulation 
approximations  erode  the  stability  margins  sufficiently 
to  require  a  significant  change  in  pilot  gain,  or  even 
more  dramatically  to  change  the  overall  control 


strategy',  then  the  simulator  is  no  longer  appropriate  for 
flight  clearance  even  though  it  will  often  be  fixable 
using  different  control  techniques.  Typically  pilots  will 
initially  modify  their  control  activity,  but  if  this  fails 
thev  may  overcontrol  or  adopt  a  form  of  open  loop 
control  using  pulse  or  small  step  inputs  and  waiting  to 
see  the  steady  state  change.  This  approach  is  a  valid 
way  of  achieving  stability  as  open  loop  systems  cannot 
be  driven  unstable  (although  they  may  be  naturally 
unstable).  These  methods  of  compensating  for  deficien¬ 
cies  may  lead  to  totally  different  control  strategies 
compared  with  those  used  in  flight  although  they  may 
give  similar  performance.  Pilots  will  often  compare  this 
with  playing  computer  games  where  dynamic  motion 
feedback  is  usually  missing.  In  most  cases  the  lower 
stability  margins  in  a  simulator  make  a  given  task  more 
difficult  than  it  is  in  the  aircraft.  However  this  will  not 
necessarily  be  true  where  pilots  change  their  control 
strategy1.  There  are  also  cases  where  feedback  from 
motion  cues  make  the  aircraft  difficult  to  control2  and 
this  would  make  a  fixed  base  simulator  easier  to  fly  than 
the  aircraft.  This  latter  effect  can  be  particularly  signifi¬ 
cant  in  advanced  flight  control  aircraft  where  large 
control  surface  movements  and  complex  interactions 
can  be  demanded  for  'normal'  control  inputs. 

In  the  following  sections  the  effects  of  various  simula¬ 
tion  approximations  that  influence  dynamic  behaviour 
are  discussed  in  relation  to  a  variety  of  tasks.  There  are 
three  main  topics  considered: 

a  computing  time  delays  and  phase  lags 

b  motion  feedback 

c  harmonisation  of  various  cues 

3.1. 1.1  Computing  delays 

Computation  of  vehicle  and  system  states  and  the  visual 
scene  requires  time  and  introduces  delays  that  are  not 
present  in  flight.  Pilots  will  not  usually  be  able  to 
identify  that  lags  are  present  unless  they  become  very 
large,  but  these  delays  will  reduce  the  stability  margin 
on  any  simulator  compared  with  that  in  flight.  Delays 
will  depend  on  many  factors  including  computing 
speed,  computer  configuration,  operating  system,  soft- 
W'are  configuration,  hardware  interfacing,  etc.  It  is 
important  to  identify  actual  delays  as  both  the  total 
delays  and  the  relative  delays  between  different  cueing 
systems  (see  the  later  section  on  harmonisation)  are 
important.  Current  requirements3  for  Civil  Training 
Simulators  to  meet  the  highest  certification  standards 
are 

T50  milliseconds  or  less  after  airplane  response1, 
for  Visual,  Motion  and  Instrument  systems 
response  to  an  abrupt  pilot  controller  input,  or 
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'a  transport  delay*  of  '150  milliseconds  or  less  after 
control  movement' 

This  delay  relates  to  the  rather  gentle  manoeuvres  made 
by  civil  airliners  during  take  off.  cruise  and  landing.  For 
more  agile  manoeuvring  by  military  aircraft  the  maxi¬ 
mum  acceptable  delay  will  be  less  than  150ms.  Military 
aircraft  specifications4-5  require  delays  in  aircraft 
systems  of  less  than  100ms.  Many  aircraft  models  will 
include  delays  of  various  kinds  already  present  in  the 
aircraft  systems.  Control  stability  is  critically  dependent 
on  total  delay  times  and  this  means  that  the  total  of 
aircraft  and  simulator  delays  should  not  be  allowed  to 
exceed  100ms  by  a  significant  amount.  A  suitable  target 
would  be  significantly  less  than  100ms  additional  delay 
from  simulation  systems.  This  may  prove  a  difficult 
task,  but  it  is  a  worthwhile  target  if  the  effects  of  delays 
are  to  be  minimised.  If  the  additional  delay  is  greater 
than  50ms  then  the  effects  on  piloting  strategy-  and 
performance  must  be  carefully  evaluated.  In  general  the 
presence  of  motion  cues  with  delays  less  than  visual 
cues  will  reduce  the  effects  of  visual  system  delays. 

Because  pilots’  responses  are  limited  to  frequencies 
below  about  2Hz  it  is  possible  to  simplify  aircraft 
system  models,  e.g.  by  eliminating  high  frequency 
structural  filters,  and  thereby  reduce  total  time  delays  or 
phase  lags.  These  reductions  can  be  used  to  offset  some 
of  the  delays  and  phase  lags  arising  from  simulation 
systems.  This  technique  was  used  in  the  NASA  Space 
Shuttle  simulation  described  in  Appendix  B. 

Compensation  algorithms  are  commonly  used  to  allevi¬ 
ate  the  effects  of  time  delays,  particularly  in  visual 
scene  generation  and  display  systems.  Typically  these 
use  the  current  calculated  rates  of  movement  to  predict 
a  position  error  at  the  end  of  the  computation  and 
display  process.  (It  is  not  desirable  to  apply  filters  to 
position  signals  alone  as  the  process  of  differentiation 
introduces  delays  in  the  derived  rates  which  are  not 
present  in  the  directly  calculated  signals.)  Whilst 
judicial  use  of  such  algorithms  can  produce  significant 
improvements,  they  can  never  be  more  than  a  simple 
prediction  and  can  even  increase  errors  following  abrupt 
movements.  Time  lost  through  systems  delays  and  lags 
can  only  be  partially  recovered  and  every  effort  must  be 
made  to  minimise  such  delays.  In  general  the  effects  of 
any  delays  in  the  response  of  cueing  systems  to  pilot 
inputs  that  are  over  and  above  delays  inherent  in  the 
aircraft  should  be  evaluated  even  if  compensation 
algorithms  are  operating. 


'Transport  delay  is  the  total  simulator  system 
processing  time  required  for  an  input  signal  from  a  pilot 
primary  control  until  motion  system,  visual  system,  or 
instrument  response.  It  is  the  overall  time  delay  incurred 
from  signal  input  until  output  response.  It  does  not 
include  the  characteristic  delay  of  the  airplane 
simulated.' 


3.1. 1.2  Motion  feedback 

It  is  possible  to  use  visual  cues  to  derive  rates  of  move¬ 
ment,  or  even  accelerations.  However,  this  requires 
differentiation  of  position  information  and  thus  there 
will  always  be  a  small  lag  relative  to  the  perception  of 
position.  In  general  visual  cues  of  angular  positions  and 
rates  are  more  obvious  them  those  for  translational 
position  and  velocities,  unless  the  aircraft  is  very  close 
to  the  ground.  Height  rate  information  is  particularly 
difficult  to  obtain  from  visual  scenes  unless  the  aircraft 
is  hovering  close  to  a  vertical  object  which  is  rich  in 
detail. 

Motion  systems  provide  the  pilot  with  advanced 
warning  about  future  changes  in  aircraft  motion  that  is 
not  available  from  any  of  his  other  senses.  Sometimes 
this  information  can  be  obtained  from  cockpit  instru¬ 
ments,  but  this  often  leads  to  unrepresentative  control 
strategies,  or  is  not  a  usable  source  of  information,  e.g. 
when  hovering  and  landing  using  external  visual  cues, 
when  there  is  insufficient  time  to  scan  cockpit  instru¬ 
ments. 

Motion  systems  can  be  either  the  familiar  motion 
platform  which  moves  the  entire  cockpit,  or  'pseudo' 
motion  systems,  such  as  dynamic  seats,  which  act  upon 
parts  of  the  body  to  induce  motion  sensations.  (Large 
quasi-steady  normal  acceleration  cueing  through  'g' 
seats  and  visual  effects  are  part  of  the  cues  required  for 
situation  awareness  and  are  dealt  with  in  Section  3.1.2) 

Currently  available  platform  motion  systems  can 
adequately  represent  abrupt  and  high  frequency  motions 
such  as  buffet  or  the  consequences  of  engine  failure 
during  take  off.  The  particularly  difficult  areas  are  low- 
frequency  and  gentle  motions  such  as  controlling  a 
hovering  helicopter.  Only  motion  platforms  with 
unusually  large  translational  movements  such  as  the 
NASA  VMS6  at  Ames  Research  Center,  California. 
USA  and  the  DRA  AFS7  at  Bedford,  UK  can  provide 
effective  motion  cues  at  frequencies  down  to  around 
lrad/s  (0.16Hz). 

Dynamic  motion  seats  can  provide  another  source  of 
motion  cues.  The  DRA  has  shown  that  a  dynamic  seat 
can  provide  the  vertical  motion  cues  that  are  required  to 
perform  hovering  tasks8.  This  system  provides  cueing 
that  pilots  can  interpret  as  a  motion  cue  by  stimulating 
the  pilot's  kinaesthetic  and  touch  sensors  without  any 
real  motion,  but  it  is  not  identical  to  physical  cues  in 
flight  and  only  partly  simulates  the  wide  range  of 
sensations  experienced  in  flight.  This  makes  it  difficult 
to  be  confident  that  dynamic  seat  cues  will  be  inter¬ 
preted  consistently  in  a  wide  range  of  situations.  Until 
more  experience  is  available  it  would  be  wise  to  limit 
the  use  of  these  indirect  cues  to  those  tasks  where  pilot 
dynamic  control  is  not  critical,  i.e.  for  training  rather 
than  clearance  tasks. 

Modem  highly-damped  command  control  systems 
would  appear  to  be  less  sensitive  to  the  absence  of 
motion  feedback  to  the  pilot  since  he  is  not  required  to 
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contribute  the  degree  of  lead  compensation  that  is  aided 
by  motion  cueing.  However,  absence  of  cockpit  motion 
can  completely  mask  potential  biodynamic  feedback 
problems  resulting  from  the  large  acceleration/control- 
deflection  gradients  that  may  accompany  the  short 
response-times  that  are  characteristic  of  these  systems. 
This  problem  has  been  seen  in  the  development  of 
several  systems  that  utilised  force-controllers.  Flight  test 
results  dictated  the  need  for  filtering  of  the  controller 
output  or  reverting  to  position  controllers.  If  a  motion 
simulator  is  to  accurately  represent  this  type  of  problem, 
special  attention  must  be  given  to  the  dynamic  perform¬ 
ance  of  the  motion  system,  for  the  problem  is  likely  to 
appear  in  the  form  of  oscillation  frequencies  near  2  Hz. 
This  implies  that  the  system  geometry,  inertia,  and 
dynamics  must  be  correctly  represented,  and  also  that 
there  is  sufficient  acceleration  at  the  pilot's  position  to 
stimulate  the  problem.  Finally,  simulation  lags  must  be 
kept  as  low  as  possible  to  avoid  exacerbating  tire 
problem. 

Absence  of  motion  cues  has  been  indicted  for  failure  to 
predict  serious  Pilot  Induced  Oscillation  (PIO)  problems 
in  Fixed-based  simulation.  Reasons  for  this  include  the 
possibility  that  the  pilot  in  the  simulator  has  not  been 
motivated  to  elevate  his  control  input  gain  to  the  point 
of  inducing  dynamic  instability.  Popular  explanations 
are 

i  in  the  absence  of  normal  flight  cues,  the  pilot 
adopts  significantly  reduced  gains  in  order  to 
produce  satisfactory  performance  in  tasks 
which  are  not  severely  time-dependent,  and 

ii  the  increased  pilot  gain  induced  by  the  often 
severe  penalties  of  unsuccessful  performance  in 
flight  is  difficult  to  reproduce  in  the  artificial 
world  of  the  simulator. 

Recent  studies  of  this  problem  have  resulted  in  the 
suggestion  that  exaggerated  time  dependencies  and 
performance  targets,  as  well  as  elements  of  surprise,  be 
added  to  the  simulator  tasks  to  increase  the  chances  of 
exposing  PIO  tendencies.  This  technique  has  been  used 
successfully  in  recent  studies  at  DRA,  Bedford7  using 
the  Large  Motion  System  of  tire  AFS. 

As  discussed  in  Appendix  B.  acceptance  tests  of  the 
NASA  Space  Shuttle  landing  control  systems  were 
conducted  in  motion-based  simulation  more  than  a 
decade  ago.  In  that  case,  the  customer.  NASA,  was  not 
willing  to  accept  the  results  of  fixed-base  simulation 
assessments  as  adequate  support  for  the  necessary 
design  decisions.  Cockpit  motion  proved  valuable  to  the 
assessments,  and  has  been  sought  by  the  program  in  all 
similar  assessments  since  that  time.  At  the  request  of  the 
Air  Force,  the  NASA  VMS  facility  is  also  being  used  in 
earlv  developmental  acceptance  tests  of  a  large  military' 
transport  aircraft  that  utilises  a  fully  electronic 
command  control  system.  At  first  sight,  it  appears  that 
this  new  simulation  is  identifying  undesirable  lurching 
of  the  cockpit  that  leads  to  destabilizing  pilot  control 


inputs,  behaviour  that  was  not  exposed  in  a  fixed  base 
simulator. 

3.1.1.3  Harmonisation  of  cues 

In  the  past,  arguments  have  been  made  for  matching  the 
lag  characteristics  of  visual  and  motion  systems,  but 
current  evidence  would  support  minimising  the  lags 
independently,  never  lengthening  a  delay  to  match  the 
slower  element.  However,  motion  cues  must  occur  with 
or  before  visual  cues  if  they  are  to  provide  advanced 
warning  about  future  changes  in  aircraft  motion. 

3.1.2  Situation  cueing 

In  addition  to  responding  to  dynamic  cues,  pilots  also 
require  a  wide  range  of  situation  information  from 
which  to  plan  and  implement  changes  in  flight  state.  In 
broad  terms,  situation  information  identifies  the  current 
state  of  the  aircraft  and  its  environment  so  that  a  pilot 
can  decide  whether  or  not  to  initiate  a  change.  This  may 
be  an  urgent  need,  e.g.  collision  avoidance,  or  longer 
term.  e.g.  approach  of  a  navigation  way  point. 
Information  comes  from  a  wide  range  of  different 
sources  in  flight  and  there  is  usually  significant  redun¬ 
dancy  of  information,  which  is  often  important  in  assist¬ 
ing  a  pilot  to  identify  and  confirm  situation  changes 
rapidly.  Typical  sources  are  the  visual  scene,  motion, 
cockpit  instruments,  sounds,  vibration,  and,  on  aircraft 
with  direct  linkage  to  flying  controls,  control  loads. 
Because  of  the  redundancy  of  information  and  the  need 
for  consistency  with  flight  situations  it  is  important  that 
situation  cues  are  adequately  harmonised  to  prevent 
confusion  from  apparently  conflicting  cues. 

The  importance  of  particular  situation  cues  is  tied 
directly  to  the  tasks  to  be  undertaken  on  the  simulator. 
However,  cockpit  instruments  are  needed  for  all  tasks, 
and  visual  scenes  for  nearly  all  tasks.  Motion,  sound, 
etc.  should  be  chosen  after  appropriate  analysis  of  the 
tasks. 

Although  they  will  always  be  available  as  important 
sources  of  information,  cockpit  instruments  should  not 
be  used  as  an  alternative  to  visual,  motion  or  other  cues 
if  this  significantly  delays  recognition  of  important 
situation  changes  and  thus  impairs  response  perform¬ 
ance.  An  obvious  example  would  be  to  use  instruments 
to  identify  an  engine  failure  during  take  off.  Tire  pilot 
will  eventually  notice  that  his  airspeed  is  not  increasing 
normally  and  that  there  are  other  indications,  perhaps 
including  an  engine  warning,  but  the  situation  will  need 
to  be  verified  before  taking  appropriate  corrective 
action.  If  motion  and  visual  scene  are  present  then 
powerful  yawing  motion  cues  will  alert  him  to  the 
problem  immediately  in  an  unambiguous  way  that  is 
very  similar  to  that  on  an  aircraft.  It  is  also  important 
that  cockpit  instrument  displays  are  configured  as  they 
appear  in  the  aircraft  and  have  the  same  dynamic 
response,  e.g.  height  rate  instruments  should  include 
representative  pneumatic  lags  and  any  accelerometer 
signal  should  be  calculated  at  the  correct  location  of  the 
accelerometer  relative  to  the  aircraft  centre  of  gravity. 


9 


Requirements  for  all  aspects  of  visual  cueing  systems, 
such  as  field  of  view  (FOV),  resolution,  contrast, 
brightness,  and  scene  details,  including  cultural  features, 
texture  and  contouring,  are  closely  dependent  on  the 
tasks.  In  most  cases  pilots  will  be  the  final  arbiters  on 
the  adequacy  of  a  visual  system  for  chosen  tasks, 
although  in  all  cases  the  task  must  be  clearly  identified 
to  the  pilot  and  the  possibilities  of  changing  tasks  to 
accomplish  the  required  certification  objective  should 
be  considered  before  rejecting  a  particular  simulator 
configuration.  The  use  of  artificial  visual  features  can  be 
acceptable  as  long  as  it  is  clear  that  they  stimulate  the 
same  control  response  and  strategy  from  pilots  as  would 
occur  in  flight. 

Simulation  of  the  flight  tasks  of  typical  transport- 
categoiy  aircraft  can  be  adequate  with  a  limited  and 
fixed  forward  FOV.  A  typical  50°  x  35°  window  will 
support  the  tasks  of  take-off  and  final  approach.  An 
additional  window  providing  more  lateral  view  is 
helpful  in  approach  pattern  turns,  but  seldom  can  be 
considered  necessary.  The  simplicity  of  this  requirement 
is  due  to  the  fact  that  the  predicted  flight  path  of  interest 
is  almost  always  constrained  to  the  limited  forward  field 
of  view.  The  required  FOV  grows  larger  for  simulations 
of  the  landing  flare  and  touchdown,  of  tactical  manoeu¬ 
vres  involving  high  turn  rates  near  the  ground,  and  in 
air-to-air  combat.  In  the  simulation  of  hovering 
manoeuvres  with  helicopters  or  VTOL  aircraft,  FOV 
requirements  are  also  greater  because  of  the  essentially 
omnidirectional  capabilities  of  these  aircraft.  The 
instantaneous  flight  path  is  often  displaced  as  much  as 
90°  laterally  or  vertically  from  the  aircraft's  reference 
axis. 

The  desired  level  of  scene  detail  is  an  inverse  function 
of  the  distance  between  the  viewer  and  the  terrain 
objects.  Some  minimum  value  of  "spatial  density" 
(contrasts  per  unit  viewing  area)  is  necessary  to  effec¬ 
tively  define  a  ground  plane  or  other  surface.  Thus, 
simulations  of  hovering  landings  or  "nap  of  the  earth 
(NOE)"  flight  most  challenge  the  technology',  and 
benefit  greatly  from  the  more  recently  developed 
capability  to  represent  surface  textures  in  photographic 
detail  at  close  range.  Precision  hover  in  the  presence  of 
even  a  well-detailed  simulated  ground  plane  still 
appears  more  difficult  in  simulation  without  extensive 
motion  than  it  is  in  flight,  but  with  the  provision  of  a 
reasonable  field  of  view  and  pseudo  motion  from  a 
dynamic  seat,  simulation  can  be  adequate  for  many 
hovering  tasks.  Important  to  the  effectiveness  of  the 
visual  simulation  is  the  provision  of  familiar  near-field 
details  that  provide  a  sense  of  scale,  and  thus  a  sense  of 
proximity.  Accurate  perception  of  self-motion  is  depen¬ 
dent  on  an  accurate  perception  of  distance  from  the 
visual  cues.  In  the  real  world,  as  a  surface  or  objects  are 
approached,  higher  and  higher  levels  of  detail  are 
perceived,  and  at  close  ranges,  normal  binocular  depth 
perception  can  become  a  factor  in  assessing  proximity. 
In  the  visual  simulation,  neither  of  these  factors  is 
normally  present.  Related  to  the  level  of  detail  is  the 


resolution  provided  by  the  visual  display  devices. 
Modem  systems  are  producing  of  the  order  of  one 
million  pixels  per  "window".  If  each  window  is  not 
expanded  to  diagonal  sizes  beyond  about  70°,  the  reso¬ 
lution  is  adequate  for  most  flight  tasks  other  than  those 
requiring  identification  of  distant  targets,  where  about  4 
times  better  resolution  is  needed. 

Some  compensation  for  deficiencies  in  downward  FOV 
and  surface  detail  when  hovering  has  been  obtained  by- 
altering  the  scene  to  include  numerous  objects  closer  to 
hovering  eye-height.  A  unique  approach  has  been  taken 
in  a  VTOL  controls  system  development  program  being 
conducted  at  NASA  Ames  Research  Center.  For  hover 
tests  with  the  YAV-8B  aeroplane,  several  arrangements 
of  target  boards  are  suspended  at  altitudes  between  50 
and  100  feet  above  the  ground.  These  "real"  visual 
scenes  are  easily  simulated  in  a  laterally  disposed  three- 
window  scene,  and  the  results  of  flight  and  simulator 
tests9  have  confirmed  sufficient  equivalence  of  the 
visual  cueing  situations  for  the  selected  tasks. 

Another  important  situation  cue  in  military  flying  is  the 
normal  acceleration,  or  'g'  force.  This  cannot  be  directly 
provided  on  simulators,  apart  from  a  few  specialist 
centrifuges  which  are  very  limited  in  other  simulation 
capabilities.  Pilots  are  usually  presented  with  some  of 
the  secondary  cues  associated  with  'g'  forces,  such  as 
inflation  of  g-suit,  dimming  and  tunnelling  of  visual 
scene,  buffet,  and  indications  on  a  'g'  meter.  Meter 
indications  are  probably  the  least  useful  in  most  circum¬ 
stances  as  pilot  attention  is  often  on  the  outside  visual 
scene  during  high  'g'  manoeuvres  and  reading  a  meter  is 
very  distracting.  Voice  readings  of  *g*  levels  may  help  if 
there  is  no  other  speech  activity. 

In  the  real  world  of  flying,  the  pilot  records  and  cata¬ 
logues,  more  or  less  automatically,  a  wealth  of  other 
situation  cues.  Good  examples  are  the  vibrations 
induced  in  aircraft  by  buffet  in  either  transonic  manoeu¬ 
vring  or  when  approaching  the  stall,  etc.  Also,  the 
feedback  to  the  stick  of  forces  on  the  aircraft  control 
surfaces  are  at  times  automatically  used  to  anticipate 
some  peculiar  aircraft  behaviour.  A  special  case  is  the 
difficulty  encountered  in  keeping  control  of  the  aircraft 
in  the  dynamic  environment  often  found  at  the  bounda¬ 
ries  of  the  flight  envelope,  e.g.  the  all  axis  accelerations 
of  a  departure  or  spin  are  critical,  and  their  careful 
examination,  in  the  correct  combination,  is  of  para¬ 
mount  importance  to  assess  or  ensure  the  ability  of 
pilots  to  cope  with  the  situation. 

The  unique  aerodynamic  noise  of  a  spinning  aircraft  at 
high  yaw  rates,  and  its  distribution  or  direction  in  the 
cockpit,  combined  with  the  side-force  acceleration,  can 
be  of  the  greatest  value  in  helping  the  pilot  to  recognise 
the  situation,  but  not  necessarily  contributing  to  its 
solution;  e.g.  the  lateral  acceleration  in  spinning  often 
prevents  the  pilot  from  getting  a  centred  position  of  the 
stick  which  is  sometimes  essential  for  recovery.  In  this 
case,  the  pilot  is  faced  with  a  conflicting  situation: 
relevant  noises  and  lateral  accelerations  are  a  clear 
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indication  of  spinning  (sometimes,  were  it  not  for  these 
cues,  it  would  be  difficult  to  distinguish  the  spin  from  a 
tight  spiral)  and  assist  spin  recovery  procedures.  The 
recovery  is  made  more  difficult  by  the  accelerations 
themselves,  the  effect  of  which  is  to  confuse  the  pilot 
about  the  position  of  the  controls  (particularly  slick 
position). 

Engine  surging  or  stalling  is  another  good  example: 
some  types  of  "soft"  stall  are  only  sensed  by  vibrations 
induced  in  the  aircraft  structure,  or  by  buffet  induced 
noise,  or  by  the  special  smell  immediately  present  in  the 
cockpit  through  the  air-conditioning  system.  At  the 
same  time,  engine  stall  can  be  mistaken  for  pneumatic 
valves  snapping  loudly  closed  or  open.  Much  in  the 
same  way,  the  rumbling  of  an  air-conditioning  pipe  can 
be  mistaken  for  an  engine  surge,  or  for  intake  buzz.  It  is 
therefore  a  special  combination  of  cues  that  allows  the 
pilot  to  recognise  a  situation,  before  relevant  instrumen¬ 
tation  is  examined  or  even  in  the  absence  of  suitable 
instrumentation. 

It  is  also  essential  to  investigate  whether  it  is  necessary 
to  reproduce  in  the  simulation  realistic  cues  with  a  high 
degree  of  fidelity,  or  whether  the  pilot  can  assess  the 
system  accurately,  knowing  that  secondary7  cues  are 
absent  or  reduced. 

3.1.3  Pilot  adaptability  and  subjective  assessments 
In  addition  to  perceiving  cues  humans  also  learn  to 
process  and  interpret  this  information  so  that  they  can 
choose  actions  that  are  most  likely  to  achieve  their 
requirements.  Pilot  training  and  practical  experience 
over  many  years  teaches  a  range  of  'standard'  responses 
to  cues  and  any  mismatch,  or  absence,  of  these  cues  in  a 
simulator  will  require  some  adaptation  of  these  learned 
responses.  If  the  differences  between  flight  and  a  simu¬ 
lator  are  sufficiently  small  then  pilots  will  easily  adapt 
to  the  system  and  achieve  similar  task  performance. 
Subjectively  they  may  find  little  difference  in  handling 
qualities  and  workload,  but  there  will  always  be  some 
difference  in  control  activity  if  the  simulation  differ¬ 
ences  are  significant  for  the  task.  Larger  differences  can 
lead  to  significant  changes  in  control  strategy  when 
compared  with  flight  and  pilots  will  usually'  find  the 
simulation  unsatisfactory'  even  though  performance  may 
be  similar  to  that  in  flight.  The  degree  of  adaptation  also 
influences  the  time  required  for  pilots  to  familiarise 
themselves  with  the  simulation. 

Relevant  measures  of  the  amount  of  adaptation  can  only 
be  determined  where  the  subject  pilots  have  established 
their  competence  with  the  class  of  aircraft  and  tasks  to 
be  simulated.  Pilots  without  a  high  level  of  relevant 
competence  will  not  be  able  to  judge  the  validity  of  a 
simulation.  Although  they  will  usually  be  quick  to  learn 
how  to  manage  the  simulation,  they  will  have  no  way  of 
knowing  whether  their  control  responses  and  strategies 
are  representative  of  flight.  In  this  regard,  relevant 
competence  may  be  provided  by  test  pilot  skills  in 
evaluation  as  much  as  operational  experience. 


The  following  comments  stem  from  opportunities  io 
conduct  or  to  witness  many  simulator  assessments  of 
novel  aircraft  configurations,  or  new  control  or  guidance 
concepts.  Overwhelming  factors  in  the  effectiveness  of 
these  evaluations  have  been  the  background  each 
evaluation  pilot  has  brought  to  the  simulator  in  terms  of 
flight  and  flight  simulation  experience,  his  experience  in 
the  particular  simulation  facility,  and  the  opportunity  he 
has  been  given  to  become  fully  educated  and  trained 
regarding  the  system  he  is  evaluating.  The  effective 
simulator  pilot  appreciates  that  in  the  presence  of  cue 
degradation  in  simulation,  aircraft  response  characteris¬ 
tics  will  be  harder  to  identify  than  in  flight,  thus  the 
familiarisation  process  will  take  longer.  He  will  insist 
on  being  given  the  time  to  achieve  a  performance 
plateaux.  He  will  not  accept  performance  levels  signifi¬ 
cantly  lower  than  he  would  accept  in  flight.  Because  he 
has  operational  or  test  experience  in  related  flight  and 
simulator  piloting,  he  will  have  a  reasonable  basis  for 
extrapolating  his  simulator  assessments  to  a  flight 
environment.  He  should  be  able  to  qualify  his  confi¬ 
dence  in  the  evaluations  with  his  own  assessment  of  the 
fidelity  of  the  simulation.  He  believes  that  simulation, 
properly  utilised,  is  a  valuable  tool  in  the  development 
and  acceptance  of  aircraft  and  aircraft  systems.  The  test 
conductor  is  seriously  constrained  if  a  pilot  with  this 
experience  and  motivation  is  not  among  his  evaluators. 

There  is  every  reason  to  expect  that  when  the  pilot  is 
first  introduced  to  the  simulation  of  a  familiar  aircraft, 
though  it  is  accurate  in  every  sense  except  for  reduced 
motion  feedback,  simplified  visual  cues,  and  probably 
increased  latency,  he  will  have  some  difficulty  subjec¬ 
tively  accepting  it  as  the  equivalent  of  flight.  In  all 
probability,  pilot  workload  will  be  higher,  and  perform¬ 
ance  reduced.  It  is  also  probable  that  the  pilot  will  not 
perceive  reduced  levels  or  absence  of  motion  feedback 
as  the  source  of  his  difficulties,  since  these  cues  are 
normally  perceived  at  the  sub-conscious  level.  His 
continued  exercise  of  the  simulation  will  result  in 
improved  performance  and  an  increased  sense  of  simu¬ 
lation  fidelity  as  he  adapts  to  the  simulation  motion  cues 
and,  again  sub-consciously,  develops  compensatory 
precognitive  control  techniques.  It  has  been  noted  that 
in  most  cases  where  the  pilot's  first  flight  in  the  aircraft 
follows  many  hours  in  its  ground-based  simulation,  the 
simulation  is  perceived  as  having  been  an  accurate 
predictor  of  the  characteristics  of  the  aircraft,  though  the 
difficulty  of  critical  tasks  may  tend  to  be  less  in  the  real 
flight  environment.  This  dependence  of  subjectively 
sensed  simulator  fidelity  on  the  order  of  the  flight  and 
simulator  experiences  is  understandable.  In  the  flight-to- 
simulator  transfer,  cues  important  to  the  pilot's  mode  of 
control  have  been  degraded  or  eliminated,  resulting  in 
some  level  of  initial  confusion  and  performance  degra¬ 
dation  relative  to  flight.  The  reverse  transfer  simply 
adds  additional  cues  in  tasks  in  which  the  pilot  quite 
likely  has  developed  a  comfortable  level  of  performance 
in  the  simulator. 
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Workload,  or  stress  is  another  factor  limiting  human 
performance  and  it  is  important  that  simulation  has  a 
similar  balance  of  different  contributors  to  the  overall 
workload.  There  are  often  comments  on  the  inability  of 
simulation  to  create  the  ultimate  levels  of  fear  and 
apprehension  that  may  be  experienced  in  life  threaten¬ 
ing  situations  because  the  ultimate  sanction  of  death  is 
not  present.  This  is  not  relevant  to  flight  clearance 
activities  as  they  deal  with  situations  and  means  of 
ensuring  safety,  where  the  main  incentive  for  pilots  is  to 
avoid  stigma  from  displaying  poor  airmanship. 
However,  it  is  important  to  represent  the  total  workload 
experienced  by  the  aircrew  for  any  simulated  task  as  the 
ability  of  the  crew  to  respond  in  critical  situations  will 
often  be  constrained  by  the  presence  of  other  necessary 
tasks.  It  is  strongly  recommended  that  workload  should 
be  compared  between  flight  and  simulation  when  vali¬ 
dating  a  simulator. 

In  his  assessment  of  a  simulated  aircraft's  basic  control 
system  and  handling  qualities,  the  pilot  will  normally 
extrapolate  his  experience  to  flight  and  the  operational 
distractions  he  expects  to  encounter.  If  the  peripheral 
tasks  are  unique  and  unfamiliar,  they  should  also  be 
included  in  the  simulation.  If  the  real  environment 
requires  frequent  distracting  interactions  between  crew 
members  and/or  outside  elements  such  as  traffic  control 
or  tactical  command,  consideration  should  be  given  to 
their  simulation.  In  the  evaluation  of  systems  in  tasks 
peripheral  to  the  primary  flight  control  objectives,  the 
simulation  should  include  stressful  versions  of  the  basic 
flight  task.  This  is  particularly  important  in  those  cases 
involving  the  scanning  of  information  sources  away 
from  the  basic  flight  displays  and  manual  input  to  the 
systems.  Consideration  should  be  given  to  the  impor¬ 
tance  of  cockpit  motion  and  vibration  environments  in 
the  evaluation  of  such  systems. 

In  the  end,  only  the  dedicated  pilot  can  give  the  best 
evaluation  of  how'  many  of  the  validation  goals  have 
been  achieved  through  simulation.  To  offer  consistency- 
in  the  contribution  given,  it  is  therefore  necessary-  that 
the  pilot  employed  in  the  simulation  be  familiar  with 
both  the  real  system  and  the  simulator,  so  that  the 
degree  of  his  conscious  compensation  is  adequate  to 
obtain  both  die  desired  validation  results,  and  weighted 
judgements  on  its  validity. 

There  is  no  doubt  that  flight  simulation  is  a  valid  tool 
for  acceptance  testing:  it  may  be  the  only  way  to  gain 
experience  in  some  phases  of  a  programme.  Once  vali¬ 
dated  with  flight  test  findings,  the  simulation  can  be  a 
valuable  tool  in  the  clearance  process  and,  in  some 
instances,  can  be  the  only  tool  used  to  get  acceptance  of 
a  system.  The  contribution  of  the  pilot  with  dedicated 
training  is,  especially  in  this  last  case,  as  important  as  it 
is  in  flight  testing.  In  this  respect,  good  progress  can  be 
made  only  if  the  engineer-pilot  relationship  is  tightly- 
linked.  as  is  usual  in  flight  testing. 

The  use  of  research  and  engineering  simulation  has 
recently  given  outstanding  results  in  the  fields  of  in¬ 


house  certification,  customer  acceptance  of  modifica¬ 
tion.  and  new  issues  of  software.  In  this  respect,  the  use 
of  simulation  has  been  extremely  productive  in  the 
fields  of  flight  control,  avionics,  and  display  optimisa¬ 
tion.  Also,  it  has  been  instrumental  in  getting  customer 
acceptance  of  the  intended  path  to  be  followed  to 
implement  changes  or  solve  problems  in  all  three  areas, 
and  so  to  obtain  progress  payments. 

3.2  The  Vehicle  Model 

For  many  years  the  creation  of  mathematical  models  of 
aircraft  has  been  an  important  part  of  the  process  of 
development  and  certification/acceptance.  Models  (of 
various  degrees  of  complexity)  have  been  used  to 
predict  flight  characteristics  ahead  of  die  flight  test 
programme,  and  to  investigate  anomalies  discovered  in 
flight.  They  have  also  been  used  to  predict  performance 
in  situations  where  flight  testing  w'ould  have  been 
impractical,  too  hazardous,  too  time  consuming,  or  too 
expensive  (e.g.  complex  failure  modes,  including 
mechanical  failures,  rare  and  extreme  meteorological 
and  environmental  conditions,  or  acquiring  the  large 
amounts  of  data  required  for  the  statistical  analysis  of, 
for  example,  autoland  performance).  Many  of  these 
models  have  been  developed  using  the  fullest  possible 
descriptions  of  die  vehicle  and  its  control  systems  to 
ensure  the  highest  possible  fidelity.  Such  models  often 
cannot  be  computed  in  real  time  despite  restricting  them 
to  limited  flight  regimes  or  aircraft  configurations. 
However  this  is  not  a  problem  when  a  human  pilot  is  not 
an  active  participant  in  the  system. 

The  fundamental  aerodynamic  and  physical  equations 
used  in  these  sophisticated  non-real-time  models  are  an 
essential  guide  to  the  forms  of  model  structure  that 
should  be  developed  for  real-time  simulation. 

In  piloted  flight  simulation,  where  the  pilot  is  a  neces¬ 
sary  participant,  models  must  run  in  real  time  with 
typical  computational  times  of  between  1  and  10 
milliseconds  within  total  cycle  times  of  2  to  25 
milliseconds.  This  requires  particular  attention  to  the 
computational  efficiency  of  models  and.  sometimes, 
simplified  representation  of  systems,  e.g.  simplifying 
high  frequency  behaviour  that  is  well  outside  the  control 
response  of  a  pilot,  say  above  5Hz.  It  will  also  be  neces¬ 
sary-  to  represent  sufficient  of  the  flight  regimes  and 
configurations  for  a  pilot  to  perform  meaningful  tasks 
with  significant  deviations  from  nominal  flight  patterns. 
Such  deviations  may  well  need  to  go  into  potentially 
dangerous  flight  situations. 

Where  the  flight  mechanics  are  well  understood,  and 
where  there  is  substantial  background  experience  in 
aircraft  mathematical  modelling,  the  development  of 
models  for  piloted  simulation  should  be  relatively 
straightforward.  This  may  not  be  the  case  for  unusual 
flight  conditions  (e.g.  very  high  angles  of  attack),  novel 
aircraft  configurations  (e.g.  ASTOVL)  or  any  vehicle  or 
flight  condition  where  the  flight  mechanics  are  complex 
or  not  fully  understood  (e.g.  helicopters).  In  many  cases, 
the  simulation  models  may  be  developed  simply  and 
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directly  from  (or  alongside)  models  used  for  non  real¬ 
time  applications.  But  even  if  circumstances  dictate  that 
simulation  models  are  developed  "from  scratch",  verifi¬ 
cation  of  the  responses  of  the  simulation  model  can  be 
achieved  in  part  by  comparison  with  available  non  real¬ 
time  models.  Furthermore,  as  additional  wind  tunnel, 
engine  test  and  flight  test  data  become  available,  similar 
validation  checks  and  refinements  can  be  made  to  both 
non-real-time  and  real-time  models. 

The  specific  requirements  of  model  verification  should 
be  given  due  weight  when  defining  the  envelope  and 
configurations  which  are  to  be  tested  either  in  flight  or 
in  the  wind  tunnel.  During  the  early  stages  of  the  devel¬ 
opment  programme,  data  for  model  development  or 
verification  will  be  sparse,  but  there  may  still  be  a 
requirement  for  an  "adequate"  model  for  piloted  simu¬ 
lation  tests.  In  this  situation,  theoretical  predictions, 
engineering  judgement,  and  experience  from  previous 
aircraft  will  need  to  play  a  part.  But  the  limitations  of 
the  "immature"  models  which  can  be  developed  at  this 
stage  of  the  project  must  be  recognised  and  accepted. 
Such  models  must  be  used  with  caution,  for  "broad¬ 
brush"  or  exploratory  studies  only.  They  should  be 
checked  and  refined  as  soon  as  opportunities  arise,  and 
effort  invested  in  defining  and  implementing  such 
models  in  a  modular  way,  to  facilitate  future  refine¬ 
ments,  will  be  very  well  rewarded. 

As  to  the  scope  of  the  model  which  needs  to  be  devel¬ 
oped  (and  hence  by  implication  the  scope  of  the  verifi¬ 
cation  process),  the  key  consideration  must  be  the 
objectives  of  the  simulation  trials  programme.  Tims  the 
aspects  of  the  aircraft  which  are  to  be  demonstrated  for 
certification/acceptance  by  simulation  must  be  defined 
and  documented  in  advance  in  order  to  avoid  nugatory 
effort  being  expended  in  unwarranted  sophistication  of 
the  model  and  its  verification.  Tlius,  for  example,  if 
touch  down  and  ground-run  dynamics  are  to  be  certified 
by  simulation,  quite  elaborate  modelling  (and  verifica¬ 
tion)  of  the  gear  and  tyre  characteristics  would  be 
appropriate.  On  the  other  hand,  if  these  aspects  were  to 
be  certified  by  flight  tests  alone,  a  much  more  rudimen¬ 
tary  undercarriage  model  would  suffice,  and  verification 
of  this  aspect  of  the  model  would  be  a  much  less 
demanding  business. 

3.3  Systems  Modelling  and/or  Hardware 
Integration 

The  simulator  is  obliged  to  use  a  mathematical  model  to 
represent  the  aerodynamics,  engine,  undercarriage  and 
sensors  of  the  subject  aircraft.  However,  in  the  case  of 
avionics  and  equipment,  there  is  a  choice  between 
interfacing  the  simulator  to  actual  flight  hardware 
(stimulation),  creating  software  to  model  the  system 
(simulation),  or  using  aircraft  software  in  the  general 
purpose  computers  of  the  simulator  (emulation).  Each 
approach  has  its  advantages  and  disadvantages  in  terms 
both  of  implementation  and  validation. 

Using  actual  flight  standard  hardware  has  the  advantage 
that  the  true  equipment  characteristics  are  provided  in 


the  simulation  and  experience  in  the  simulator  may 
contribute  directly  to  refining  and  developing  the 
system  in  preparation  for  (or  in  support  of)  flight  trials. 
The  simulator  can  readily  keep  abreast  of,  and  check¬ 
out,  any  equipment  modifications,  since  hardware  and 
software  changes  can  be  implemented  immediately,  and 
the  simulator  will  capitalise  directly  on  the  configura¬ 
tion  control  philosophy  which  is  bound  to  be  applied  to 
the  flight-rated  equipment.  Further,  the  use  of  flight 
hardware  may  be  the  only  practical  option  if  the  details 
of  the  model  represented  by  the  embedded  software  are 
classified  and  cannot  readily  be  released  for  simulation 
or  emulation  in  a  general  purpose  computer.  On  the 
debit  side,  the  difficulties  in  interfacing  hardware  to  the 
simulator  may  be  significant,  particularly  if  the  system 
is  digital  and  needs  to  run  asynchronously  with  the 
simulation  frame  rate.  Additional  software  effort  may 
also  be  needed  to  model  sensors  and  communication 
protocols  to  the  level  of  accuracy  expected  by  the  flight 
hardware.  The  capital  cost  of  flight-rated  hardware  may 
be  a  disadvantage  and,  if  the  equipment  is  newly- 
developed,  availability  of  units  for  simulation  may  be 
problematic.  Significant  delays  may  occur  if  the  need 
for  "fixes"  or  refinements  arise  during  testing,  since 
these  tasks  will  generally  be  the  responsibility  of  a 
subcontractor  supplying  the  system.  When  assessing  the 
benefits  of  using  flight  hardware  to  help  'qualify' 
airborne  equipment  it  must  be  remembered  that, 
because  of  the  different  environment  of  the  simulator  as 
compared  with  flight  (e.g.  g-forces,  vibration,  tempera¬ 
ture),  the  simulator  can  only  address  the  functional 
certification  of  the  equipment  and  not  its  operation  in 
the  environmental  conditions  representative  of  flight. 
Finally,  the  flight  hardware  may  be  unsuitable  for  use  in 
the  simulator  because  it  has  no  facilities  either  to 
suspend  operation  while  the  simulator  is  in  "freeze",  or 
to  enter  known  failure  states  on  command  as  would  be 
needed  to  simulate  a  flight  in  various  failed  conditions. 
These  last  points  are  now  being  addressed  by  equipment 
manufacturers,  who  can  provide  such  facilities 
(incorporated  in  such  a  way  that  they  cannot  be  evoked 
when  installed  in  an  actual  aircraft)  in  accordance  with 
the  ARINC  standard  610.  However,  to  date  such  facili¬ 
ties  (developed  for  application  in  training  simulators) 
have  only  been  implemented  for  civil  aircraft. 

One  advantage  of  'simulating'  the  system  (i.e.  writing 
software  specifically  to  represent  the  airborne  equip¬ 
ment  within  the  simulation  computers)  is  that  the  simu¬ 
lator  development  team  have  direct  control  over  the 
development  of  the  representations.  The  level  of  detail 
embodied,  and  the  sophistication  of  the  modelling  of 
failure  states,  can  be  managed  to  suit  the  needs  of  the 
simulator  trials  programme.  On  the  debit  side,  the  effort 
required  to  develop  an  adequate  representation  of 
complex  airborne  equipment  may  be  totally  prohibitive 
(perhaps  10's  or  even  100's  of  man-years  for  a  full  flight 
management  system).  Furthermore,  data  which 
adequately  defines  the  performance  and  detailed 
responses  of  the  system  may  be  sparse  and  inadequate. 
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This  is  particularly  likely  to  be  true  of  failure  modes. 
These  factors  will  not  only  hinder  development  of  the 
simulation  software,  but  also  possibly  preclude  adequate 
verification  of  the  end  product. 

The  third  option,  that  of  'emulating'  the  system  by  using 
the  aircraft  software  in  the  general  purpose  computers  of 
the  simulator,  has  been  made  possible  by  the  increasing 
use  of  high  level  languages  in  airborne  systems  as  a 
consequence  of  the  US  DoD  mandating  the  use  of  the 
ADA  programming  language  for  all  flight  software. 
This  approach  has  major  advantages,  especially  when 
considering  the  ever  increasing  size  and  complexity  of 
modem  airborne  software,  whilst  avoiding  many  of  the 
disadvantages  of  the  other  two  approaches.  Complex 
software  systems  can  be  installed  and  verified  in  a 
matter  of  hours,  and  additional  features  to  cover  failure 
modes  or  simulation  specific  functions  can  be  readily 
introduced.  Further,  interfacing  the  aircraft  software 
with  the  simulation  software  is  often  easier  than  inter¬ 
facing  the  equivalent  hardware.  On  the  debit  side,  the 
difficulties  in  modelling  the  flight  computer  hardware 
and  interfaces  may  be  significant,  in  particular  timing 
issues  associated  with  its  internal  and  external  commu¬ 
nications.  This  is  particularly  true  if  the  hardware  has 
multiple  CPUs  with  software  communicating  between 
CPUs  during  any  frame-time. 

Thus  the  choice  between  flight  hardware,  simulator 
software  or  aircraft  software  is  unlikely  to  be  straight¬ 
forward.  The  more  complex  the  software  and/or  the 
hardware,  the  more  compelling  becomes  the  argument 
for  using  the  actual  flight  software  and/or  hardware  in 
the  simulator.  The  difficulties  of  simulating  sensors  to 
provide  appropriate  signal  inputs,  and  of  simulating 
flight  hardware  characteristics  to  provide  the  appropri¬ 
ate  host  environment  for  flight  software,  e.g.  bus  traffic, 
timing,  etc.,  should  not  be  underestimated.  Each  case 
must  be  considered  on  its  merits,  but  the  requirements 
and  objectives  of  the  simulation  must  remain  paramount 
and  be  clearly  defined.  Where  modelling  uncertainties 
remain,  and  are  of  concern  to  the  certification  authori¬ 
ties,  a  measure  of  the  significance  of  the  uncertainty 
may  be  obtained  by  varying  the  critical  parameters. 

Because  they  can  be  examined  in  safety,  very  compre¬ 
hensive  evaluations  of  system  failure  modes  and  tran¬ 
sients  can  be  conducted  in  simulation  of  the  most 
critical  flight  conditions.  Tasks  can  be  imposed  upon  the 
pilot  in  any  rational  combination  of  primary  tasks 
together  with  appropriate  secondary'  tasks.  It  has  been 
generally  observed  that  in  the  fixed  base  cockpit, 
recognition  and  control  of  failure  transients  are 
inhibited,  resulting  in  overly  pessimistic  assessments  of 
the  control  problem  induced  by  failures.  It  is  strongly 
recommended  that  simulator  assessments  involving 
failure  transients  include  cockpit  motion  capable  of 
providing  at  least  an  alerting  function. 

3.4  Scenario  and  Environmental  Modelling 

Simulation  offers  opportunities  to  test  aircraft  and 
systems  concepts  in  environments  and  scenarios  that  are 


impractical  or  impossible  to  establish  in  flight  tests. 
Since  these  "worst  case"  conditions  may  define  the 
acceptability  of  the  subject  system,  the  fidelity  of  their 
simulation  is  of  prime  concern.  Winds,  turbulence  and 
visibility  are  factors  directly  challenging  the  pilot  and 
the  vehicle.  These  and  other  operationally  induced 
distractions  and  stress  can  add  to  the  validity  of  the 
simulator  assessments,  especially  those  of  peripheral 
systems  associated  with  navigation,  communications  or 
weapons.  However,  because  attenuation  of  cueing  in 
simulation  can  exaggerate  the  basic  flight  control 
workload,  care  must  be  taken  to  avoid  scenarios  that  for 
the  same  reason  further  elevate  tasks  to  improbable  and 
unrealistic  levels  of  difficulty. 

3.4.1  Turbulence  and  Winds 

Long-accepted  methods  of  modelling  turbulence  and  its 
effects  on  aircraft  response  appear  to  be  adequate  for 
most  evaluation  activities  involving  fixed-wing  aircraft. 
It  should  be  noted,  however,  that  the  control  tasks 
presented  to  the  pilot  by  simulated  turbulence  tend  to  be 
exaggerated,  particularly  in  the  total  absence  of  cockpit 
motion.  With  conventional  motion  systems,  the  high 
frequency  portions  of  the  turbulence  can  be  sensed,  but 
flight-path  disturbances  due  to  the  lower  frequency 
components  are  still  not  perceived  in  the  normal  sense. 
This  could  explain  the  low  opinion  pilots  hold  regarding 
the  ride  quality  provided  by  simulated  turbulence. 
Conventionally  modelled  turbulence  presented  in  the 
context  of  the  large  motion  envelope  of  the  NASA  VMS 
has  been  accepted  as  realistic. 

Representations  of  winds,  and  discrete  low-frequency 
wind  variations  more  commonly  referred  to  as  shears, 
are  most  important  for  the  simulation  of  landing  and 
take-off  manoeuvres.  To  present  the  critical  cross-wind 
landing  task  with  fidelity  requires  effective  modelling  of 
the  winds  and  turbulence,  as  well  as  the  aircraft's 
landing  gear.  It  is  helpful  to  include  the  normal  wind 
velocity  gradient  with  altitude.  In  general,  "cross- 
control"  manoeuvres  present  exaggerated  difficulty  in 
the  absence  of  motion  cueing,  so  it  is  especially  impor¬ 
tant  that  the  evaluation  pilot  be  given  the  opportunity  to 
familiarise  himself  w  ith  the  task. 

3.4.2  Tactical  Scenarios 

The  newer  visual  simulation  capabilities  coupled  with 
large  capacity  computer  facilities  provide  the  means  to 
create  very  elaborate  tactical  combat  scenarios  involv¬ 
ing  several  aircraft.  In  these  situations,  the  piloting  task 
can  become  a  contest;  an  engrossing  competitive  game. 
Unfortunately,  particularly  if  the  scenario  does  not 
relate  strongly  to  his  past  experience,  the  evaluation 
pilot  may  sometimes  fail  to  give  appropriate  considera¬ 
tion  to  the  level  of  fidelity  of  the  task  presented. 

3.5  Software  Considerations 

The  validation  process  is  essentially  an  assessment  of 
the  adequacy  of  the  total  simulation  for  the  intended 
certification  tasks.  Thus  it  strictly  addresses  the  observ¬ 
able  performance  of  the  hardw  are  and  softw  are  which 
together  provide  the  complete  simulation.  Poorly 
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designed  or  engineered  hardware  may  possibiy  achieve 
adequate  performance  on  occasions,  but  it  may  be 
"temperamental”,  yieiding  results  which  cannot  be 
repeated  with  confidence,  or  even  failing  totally. 
Validation  of  such  equipment  (if  possible)  is  slow, 
frustrating  and  very'  costly.  Fortunately,  the  importance 
of  good  design  and  engineering  of  hardw  are  is  generally 
well  understood  and  such  difficulties  are  unlikely  to 
arise.  Sadly,  the  importance  of  good  software  design 
and  software  development  practices  is  less  widely 
appreciated.  But  the  scope  for  shortcomings  in  these 
aspects  to  cause  obscure  malfunctions  which  can 
frustrate  or  undermine  validation  is  far  greater  than  for 
hardware.  Thus  effort  invested  in  softw'are  design  and  in 
good  software  development  practices  will  be  repaid  in 
the  course  of  validation.  Good  software  design  will 
enforce  a  modular  structure.  Modules  can  be  tested 
individually  and  then  together.  Static  and  dynamic 
testing  identify  errors  and  build  confidence  in  the 
correct  operation  of  the  softw'are.  Testing  of  software 
cannot  prove  correct  operation,  only  the  absence  of 
errors  within  the  scope  of  the  tests.  Even  so,  it  is  a  vital 
foundation  for  the  eventual  validation  of  the  complete 
simulation. 

Strict  configuration  control  of  software  at  all  levels, 
including  programs,  data  and  system  software,  will  be 
vital  if  results  of  early  tests  are  to  retain  their  validity. 
Fortunately,  software  packages  to  enforce  configuration 
control  on  the  software  development  process  are  now 
available,  and  will  become  increasingly  essential  as  the 
volume  of  softw'are  associated  w'ith  simulations  contin¬ 
ues  to  rise. 

Some  of  the  software  required  for  the  simulation  may 
correspond  directly  with  softw'are  already  created  as  part 
of  the  aircraft  development  programme:  for  example, 
aerodynamic  or  engine  models  for  simulation  and  for 
non  real-time  development  studies,  or  software  to 
simulate  on-board  avionics  and  the  softw  are  within  the 
actual  avionic  equipment.  In  some  cases,  perhaps 
increasingly  so  with  the  emphasis  on  high  level  specifi¬ 
cation  and  design  languages  and  portable  software,  it 
may  be  possible  to  re-use  (or  modify)  the  existing  soft¬ 
ware.  Any  software  so  used  must  have  been  previously 
validated  and  be  fully  supported  by  design  and  test 
documentation.  In  other  cases,  the  requirements  of  real¬ 
time  operation,  or  the  unavailability  of  compilers  for  a 
particular  language  on  the  real-time  system,  may  mean 
that  the  software  for  the  simulator  has  to  be  developed 
from  scratch.  These  two  approaches  to  providing  soft¬ 
ware  for  the  simulator  are  quite  different  in  their  impli¬ 
cations  for  softw'are  verification.  Re-use  of  "common" 
software  substantially  reduces  the  burden  of  verifica¬ 
tion.  However,  it  is  unlikely  that  the  software  can  be  re¬ 
used  without  at  least  minor  modifications,  and  thus  a 
requirement  for  verification  will  still  remain.  In 
contrast,  if  the  simulation  software  has  been  developed 
independently,  in  a  different  language  and  using  differ¬ 
ent  compilers,  a  very  substantial  verification  effort  will 
be  needed.  Nevertheless,  there  is  one  significant  advan¬ 


tage  in  this  approach  because  the  redundancy  afforded 
by  the  independent  development  of  the  simulation  can 
provide  a  powerful  tool  to  expose  any  errors  in  either 
implementation  of  the  software. 

Choice  of  language  for  simulation  software  may  simi¬ 
larly  be  influenced  by  such  factors  as  real-time  effi¬ 
ciency,  compatibility  with  pre-existing  software, 
company  expertise,  and  availability  of  proven  (verified) 
compilers.  Eventually  Ada  is  likely  to  become  the 
obvious  (perhaps  the  only)  choice,  but  at  present  many 
of  the  above  factors  may  point  to  the  adoption  of  a  less 
sophisticated,  but  more  generally  used  languages  such 
as  FORTRAN.  However,  the  likely  lifespan  of  the 
simulation  model  should  also  be  considered.  The  most 
intensive  use  of  simulation  is  likely  to  arise  during  the 
preparation  for,  and  achievement  of,  certifica¬ 
tion/service  acceptance.  Nevertheless,  a  capable  and 
validated  simulation  is  a  substantial  investment  and 
provides  a  facility  which  should  be  of  ongoing  useful¬ 
ness  throughout  the  complete  life  of  the  project. 
Furthermore,  in  the  case  of  military  aircraft,  the 
procurement  agency  may  perhaps  in  future  require  that 
the  simulation  softw'are  be  made  available  for  applica¬ 
tion  to  training  simulators.  Equally,  in  the  case  of  civil 
aircraft,  the  manufacturer  may  be  able  to  sell  the  soft¬ 
ware  to  a  training  simulator  manufacturer,  or  possibly 
establish  some  co-operative  agreement.  The  prospect  of 
such  schemes  would  further  enhance  the  attractiveness 
of  Ada. 

A  more  detailed  check  list  for  software  configuration 
management  is  presented  in  Annex  C. 

4  VALIDATION  TECHNIQUES 

The  validation  process  is  closely  related  to  the  basic 
assumptions  made  when  designing  a  simulator.  A 
similar  logical  process  can  be  adopted  for  validating  any 
simulator  but  the  role  of  that  particular  simulator  must 
be  taken  into  account  at  each  stage.  In  many  cases  it  is 
possible  that  the  simulator  is  not  intended  to  be  repre¬ 
sentative  of  all  the  aircraft's  phases  of  flight.  However, 
it  is  often  desirable  to  allow  the  simulator  to  enter  these 
phases  so  that  a  pilot  can  transition  between  other 
phases  of  flight  in  a  continuous  manner.  In  such 
instances  the  validation  process  should  reflect  this  by 
appropriate  restrictions  on  the  validated  envelope, 
which  would  not  prohibit  the  use  of  other  parts  of  the 
envelope  for  transitioning  between  validated  areas. 

From  a  validation  point  of  view  a  simulator  has  two 
main  components:  the  mathematical  model  and  the  cues 
presented  to  the  pilot  to  allow'  interaction  w'ith  this 
model.  Usually  the  process  is  one  of  quantitative  and 
independent  validation  of  the  model  and  cues  followed 
by  closed  loop  quantitative  assessments  and  subjective 
evaluation  using  a  pilot-in-the-loop.  The  stages  in  the 
process  are  illustrated  in  Fig  2. 


15 


4.1  The  Validation  Process 

Validation  of  a  simulator  is  a  process  of  demonstrating 
by  objective  and  subjective  testing  that  the  theoretical 
model  and  the  physical  simulator  together  form  an 
adequate  representation  of  the  real  system  for  the 
intended  application  or  task.  Simulator  generated  output 
data  are  compared  to  real  system  data  obtained  by 
ground  or  flight  tests  (Fig  3).  Thus,  validation  requires 
real  world  data  for  comparison  with  simulation  results. 

In  order  to  obtain  adequate  real  world  data  which  are 
suitable  for  simulator  or  model  validation  some  form  of 
hardware  testing  must  be  conducted  e.g.  bench  tests, 
subsystem  tests  and  flight  tests.  This  can  be  a  problem 
area  and  it  is  described  in  section  4.4  in  more  detail. 
Additionally,  the  validation  process  requires  the  defini¬ 
tion  of  the  required  simulation  accuracy  (criteria  of 
acceptability)  to  decide  whether  or  not  the  validation  is 
complete.  Any  "validation  criterion"  naturally  depends 


Fig  3:  Validation  of  a  simulation  module 


Simulator  is  validated! 


Fig  2:  Stages  in  the  validation  process 


on  the  specific  application.  For  piloted 
simulators,  all  elements  of  the  models  of  the 
aircraft  and  its  systems  must  behave  so  that  the 
differences  compared  to  the  real  world  remain 
within  the  resolution  capabilities  of  the  pilot.  At 
the  same  time  the  cues  that  permit  the  pilot  to 
perceive  the  behaviour  of  the  aircraft  must  be 
adequate  for  the  task  to  be  simulated.  To  the 
pilot,  this  means  that  the  simulator  responds 
like  the  aircraft,  that  the  same  performance 
goals  can  be  achieved  with  the  same  workload 
as  in  the  aircraft,  and  that  he  uses  the  same 
control  strategy  and  responses  as  in  the  aircraft. 
On  the  other  hand,  if  simulation  is  used  for 
hardware  validation  (hardware-in-the-loop- 
simulation)  the  model  output  must  be  identical 
to  real  world  data  to  a  higher  degree  than  for 
piloted  operation.  For  example,  for  digital 
systems  the  data  timing  must  be  identical, 
otherwise  the  simulation  results  could  be  totally 
wrong.  In  addition  to  the  application  of 
objective  test  procedures  for  validation,  which 
will  be  described  in  section  4.7  in  more  detail, 
subjective  testing  procedures  (pilot  assessment) 
remain  absolutely  essential  because  models 
which  are  able  to  describe  pilots  perception 
tolerances,  especially  for  stick  force,  motion 
and  visual  cues,  are  not  available. 

Usually,  validation  of  a  simulator  system  is 
done  from  "the  bottom  up".  This  means  that  the 
subsystems  will  be  validated  or  verified  sepa¬ 
rately.  After  these  sub-validation  tests  the  total 
system  including  all  functional  interdependen¬ 
cies  will  be  validated  in  final  acceptance 
testing. 

4.2  Task  Definition 

The  overall  capability  of  a  simulator  is  a  com¬ 
bination  of  the  individual  tasks  which  it  can 
represent.  Tasks  often  overlap  to  a  great  extent 
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and  arc  fundamental  to  both  the  simulator  design  and 
validation  processes.  The  starting  point  is  a  clear 
understanding  of  the  task.  Considerations  to  be  taken 
into  account  for  tire  landing  approach  task  will  be  quite 
different  from  those  for  air-to-air  refuelling,  for 
example.  The  task  may  be  very  specific,  as  in  these  two 
examples,  or  be  of  a  more  general  nature  covering  large 
parts  of  the  aircraft's  flight  envelope,  e.g.  air  combat. 

Having  defined  the  task,  the  flow  of  activities  can 
diverge  into  two  separate  but  not  completely  independ¬ 
ent  paths.  The  first  of  these  considers  mathematical 
modelling  and  the  second,  the  cues  required. 

4.3  Modelling 

The  data  package  needed  for  simulation  depends  to  a 
large  extent  on  the  degree  of  modelling  required.  At  the 
heart  of  every  flight  simulator  is  some  form  of  aircraft 
model;  this  can  vary  from  a  simple  linearisation  to  a 
complex  non-linear  model.  Other  considerations  which 
arise  are  associated  with  the  modelling  of  aircraft  hard¬ 
ware.  Two  examples  are  given  below:  the  aircraft 
powerplant,  and  control  surface  actuation. 

In  order  to  simulate  an  engine  for  the  purposes  of  hard¬ 
ware  development,  it  is  often  necessary  to  represent  its 
many  inner  loops,  non-linearities  and  high  frequency 
characteristics.  For  a  pilot-in-the-loop  engine  model, 
considerable  simplifications  can  often  be  applied 
without  the  model  becoming  significantly  different 
within  the  pilot's  control  bandwidth.  Often,  an  equiva¬ 
lent  model  can  be  derived  which  gives  end-to-end  static 
and  dynamic  characteristics  up  to  the  highest  frequency 
relevant  to  a  pilot  (c.  2-4Hz),  which  are  very  similar  to  a 
much  more  detailed  model. 

A  similar  argument  applies  to  actuator  modelling. 
Again,  there  are  high  frequency  modes  of  operation 
within  the  actuator  control  loops  which  may  well 
simplify  to  a  second  order  or  even  first  order  transfer 
function  representing  demanded  surface  position  to 
actual  position.  However,  such  a  simplification  may  not 
be  possible  in  all  cases,  particularly  if  the  performance 
of  the  hydraulic  actuator,  e.g.  acceleration  &  rate  limits, 
is  affected  by  control  loading. 

Once  the  degree  of  modelling  of  the  component  models 
of  the  simulation  has  been  established,  a  dataset 
requirement  can  be  compiled. 

4.4  Modelling  Requirements 

For  mathematical  modelling,  data  is  required  both  to 
carry  out  the  modelling  and  then  to  validate  it.  Further, 
some  of  the  validation  data  must  be  independent  of  the 
modelling  data  to  ensure  that  any  errors  introduced 
(hiring  the  modelling  process  are  not  confirmed  during 
the  validation  process:  eg  validation  could  take  place  at 
different/intermediate  flight  conditions  as  well  as  the 
flight  conditions  used  for  defining  and  deriving  the 
model(s).  The  data  requirements  for  modelling  and 
validation  mav  also  be  different:  for  example,  modelling 
may  require  intermediate  measurements  if,  for  example, 
stick  dynamics,  actuators,  sensors  and  flight  control 
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Fig  4:  Mathematical  Model  Database  Validation 

models  are  to  be  distinguished  from  the  aerodynamic 
model,  whereas  validation  data  usually  addresses  the 
end-to-end  responses  of  the  total  vehicle.  Typically, 
validation  data  would  take  the  form  of  time  histories 
and  frequency  response  plots. 

The  data  itself  is  often  acquired  from  multiple  sources, 
such  as  wind-tunnel  results,  analysis  and  flight  test  data. 
Existing  computer  models  of  die  aircraft  or  systems 
which  may  be  real  or  non-real  time  are  often  used  for 
development.  The  data  used  for  these  models  can  be  a 
good  source  for  the  flight  simulator.  Also,  the  response 
or  outputs  from  these  models  can  provide  validation 
data  for  the  simulator  provided  they  themselves  are  first 
validated.  Fig  4  shows  the  method  by  which  this 
happens. 

But  nevertheless,  simulator  validation  requires  real 
world  data  (flight  test  data)  to  compare  the  simulated 
behaviour  with  the  real  flight  vehicle.  This  real  world 
data  package  should  be  used  as  a  reference  for  response 
comparison.  Therefore,  it  is  very  important  to  establish 
how  good  these  reference  data  are  and  how  they  are 
collected.  From  a  simulator  validation  stand-point  it  is 
necessary  to  define  the  required  data  very  well  and  to 
make  specific  flight  tests  to  get  the  right  data  which  are 
adequate  for  the  validation  process.  Up  to  now  the 
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Fig  5:  Wide  bandwidth  input  signal  for 
aircraft  or  system  excitation 

simulator  contractor  has  had  to  live  with  flight  test  data 
which  have  been  measured  during  the  prototype  and 
certification  flights.  But  these  data  are  often  not  suitable 
for  the  validation  process.  For  instance,  it  could  be 
possible  that  the  full  bandwidth  behaviour  of  the  aircraft 
is  not  excited  dining  certification  flight  testing  because 
it  is  not  required  for  this  purpose. 

Therefore,  it  is  very  important  to  define  the  input/output 
signals  to  get  data  which  can  be  used  for  simulation 
validation.  Flight  test  procedures  are  needed  to  produce 
data  which  can  be  used  for  the  special  simulator  valida¬ 
tion  process. 

Flight  test  data  must  be  corrected  so  that  the  measure¬ 
ment  errors  are  eliminated.  This  can  be  done  by 
compatibility  check  procedures  and  data  reconstruction 
methods 5. 

Suitable  flight  tests  are: 

computer  generated  input  signals  which  excite 
all  the  aircraft  dynamic  modes  (e.g.  3-2-1-1- 
signals,  Fig  5). 

frequency  sweeps  in  each  control  axis  to 
extract  control  system  frequency  response. 

special  manual  or  computer  generated 
manoeuvres. 


Another  common  component  of  the  modem 
aircraft  simulator  which  is  becoming  increas¬ 
ingly  significant  is  the  flight  control  system. 
As  we  move  towards  aircraft  with  higher 
degrees  of  relaxed  stability,  we  find  the  flight 
control  system  not  merely  augmenting  aircraft 
handling  but  having  a  gross  effect  of  stabilis¬ 
ing  a  naturally  unstable  vehicle.  Very  often, 
the  definition  of  a  flight  control  system, 
whether  analogue  or  digital,  can  be  obtained 
precisely.  Implementation  of  the  control 
system  into  a  mathematical  model  is  usually  a 
relatively  straightforward  process  compared 
with  the  aerodynamic  model,  where  data  may 
be  available  with  less  than  100%  confidence. 

Models  may  be  derived  from  flight  test  data  by 
applying  parameter  estimation  or  identification 
methods10’11’12’13.  These  methods,  which  can 
be  applied  either  in  the  time  or  frequency 
domain,  give  a  nearly  perfect  matching  of  the 
time  response  and  deliver  the  best  estimation 
of  the  relevant  model  parameters  (see  Figs  6, 
7).  An  advantage  in  using  these  methods  is  the 
elimination  of  the  need  for  visual  judgement  of 
comparisons  between  actual  and  model  time 
histories.  The  methods  include  estimation  of 
instrumentation  errors,  a  weighted  overall 
measure  of  mismatch  and  information  on  the 
confidence  levels  associated  with  individual 
parameters,  i.e.  a  measure  of  how  appropriate 
the  test  and  the  model  were  for  defining  any 
particular  parameter.  Further,  these  methods  can  be  used 
for  non-linear  systems,  systems  with  noise,  and  systems 
with  time  delays.  However,  considerable  care  and  skill 
is  needed  in  formulating  an  appropriate  model. 
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Fig  6:  Principles  of  the  Parameter 
Identification  Process 
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Fig  7:  Aircraft  Parameter  Identification 


Additionally,  the  identification  procedure  can  be  used 
for  subsystem  identification  such  as  actuators  (see  Fig 
8),  and  sensors14.  For  instance,  sensor  bias  or  incom¬ 
patibility  with  other  sensor  signals  will  also  be 
identified  in  the  process.  The  only  requirement  for  using 
these  techniques  is  to  apply  suitable  system  excitation 
inputs  such  as  the  3-2-1 -1  signal,  which  can  be 


ATTAS  -  AIRCRAFT 


generated  very  easily  by  the  pilot  or,  better  still,  by  the 
on-board  computer  system. 

It  is  strongly  recommended  that,  during  the  prototype 
testing  phase  of  an  aircraft,  flight  testing  should  be 
expanded  to  provide  the  model  parameter  data  package 
as  a  reference  for  simulation  modelling  and  validation. 
This  would  be  a  break-through  in  simplifying  the  simu¬ 
lator  modelling  and  validation  process.  At  the  same 
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Fig  8:  Actuator  System  Identification 
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time,  there  would  be  a  clear  responsibility  on  the 
aircraft  manufacturer  for  the  data  package. 

So  far  only  modelling  issues  have  been  addressed.  The 
other  consequence  of  defining  the  task  to  be  simulated 
is  that  the  cues  essential  to  represent  the  task  can  also  be 
defined. 

4.5  Cue  Requirements 

In  the  subtlest  cases,  in  order  to  determine  the  cues 
needed  for  successful  representation  of  a  flight  task,  an 
understanding  of  the  physiological  and  psychological 
processes  involved  in  carrying  out  the  task  is  needed. 
More  generally,  though,  some  basic  principles  apply. 
There  should  be  a  close  relationship  between  the  control 
feel  characteristics  in  the  simulator  and  on  the  aircraft, 
and  a  simulator  with  a  visual  system  would  be  expected 
to  provide  more  cues  than  one  without.  The  last  major 
cue  is  the  provision  of  motion  to  the  pilot  either  through 
the  seat  or  the  cockpit  itself.  Reference  15  presents 
details  of  the  closed  loop  processes  taking  place  during 
a  simulated  or  actual  flying  task.  In  general,  motion 
cues  are  more  important  for  responses  at  high 
frequencies,  and  visual  cues  dominate  in  importance  at 
low  frequencies  (0.1  Hz).  15 

Within  these  gross  assumptions  are  many  other  consid¬ 
erations.  For  example,  the  visual  cues  presented  by  a 
low  cost  visual  system  presented  to  the  pilot  over  a 
small  field  of  view  may  well  provide  adequate  cues  for 
general  handling  of  an  aircraft,  but  if  the  scene  lacks 
detail  and  texture  at  low  altitude  it  may  not  provide 
satisfactory  cues  for  a  landing  approach  -  in  particular 
for  the  flare  manoeuvre. 

Similarly,  a  particular  motion  system  may  be  unable  to 
provide  the  high  frequency  cues  to  represent  turbulence 
adequately  unless  it  has  a  high  enough  bandwidth. 
Views  on  the  need  for  motion  differ  although  the 
following  guide-lines  have  been  suggested  in  reference 
16. 

To  give  accurate  motion  sensations  requires 
high  performance  and  large  travel. 

It  is  better  to  have  no  motion  than  a  system 
which  gives  false  cueing. 

Motion  systems  are  of  most  importance  for 
representing  handling  where  stability  margins 
are  low  (i.e.  the  pilot  is  stabilising  the  vehicle), 
in  cueing  for  failure  cases  involving  a  transient 
response,  or  cueing  the  effects  of  turbulence  on 
flying  qualities. 

Small  travel  motion  system  can  be  used  with 
advantage  for  subjective  cueing  simply  to  add 
realism  to  the  simulation. 

More  recent  experience  has  also  shown  that  motion  can 
be  important  to  generate  the  cockpit  accelerations 
arising  from  the  energetic  use  of  control  surfaces  driven 
by  Active  Flight  Control  systems. 


Defining  the  drive  algorithms  for  a  motion  system  is  a 
task  which  should  be  given  careful  consideration.  Much 
has  been  published  on  the  subject  but  broadly  speaking 
the  choice  of  algorithms  will  probably  depend  on:- 

i  the  capabilities  of  the  system  in  terms  of 
number  of  degrees  of  freedom,  travels  and 
acceleration  performance, 

ii  the  type  of  aircraft  to  be  simulated  and 

iii  the  phases  of  flight  which  it  is  intended  to 
represent. 

The  validation  of  these  cues  should  initially  be  an 
objective  one  regarding  them  as  independent  sub¬ 
systems  within  the  overall  simulation.  Tests  should  be 
devised  which  prove  that  the  assumptions  made  when 
specifying  the  cueing  required  have  been  realised.  For  a 
visual  system,  the  objective  is  to  ensure  that  the  view 
seen  by  the  pilot  is  geometrically  correct  (within  the 
constraints  of  the  visual  system  hardware)  at  all  times 
and  that  the  time  delay  in  presenting  these  cues  is 
known  and  minimised.  An  arrangement  using  a 
theodolite  fixed  at  the  pilot  position  may  be  considered 
necessary  for  verifying  the  former. 

Motion  cues  can  be  validated  most  satisfactorily  by 
recording  signals  from  accelerometers  and  other  pick- 
offs  which  measure  the  motion  close  to  the  pilot's  head. 
This  combined  with  readings  taken  from  a  clinometer  to 
ensure  static  accuracy  could  be  used  to  ensure  that  the 
wash-out  algorithms  are  functioning  as  intended  and 
that  the  time  delay  between  demanded  and  achieved 
motion  are  minimal.  A  full  range  of  suitable  tests  is 
described  in  AGARD  AR14417  and  these  should  be 
combined  with  similar  tests  to  evaluate  the  combined 
dynamics  of  both  the  motion  system  and  the  drive 
algorithms,  which  include  wash  out  and  cross  coupling. 

The  tw'o  decision  paths  in  Fig  2  now  converge  as  the 
simulator  is  treated  as  a  complete  unit.  The  final 
objective  test  before  embarking  on  pilot-in-the-loop 
assessment  is  to  check  that  the  end-to-end  frequency 
responses  and  time  delays  are  acceptable  when  the 
simulator  is  operating  as  a  complete  system.  These 
responses  and  delays  are  a  combination  of  the  behaviour 
of  hardware  components  and  software  transport  delays. 
Good  update  rates  for  software  should  always  be  sought, 
particularly  as  the  trend  towards  distributed  processing 
means  that  the  flow'  of  data  may  pass  through  a  number 
of  different  processors  before  affecting  the  cues  sensed 
by  the  pilot. 

Reference  18  states  that  "accumulated  time  delays  from 
a  variety  of  simulator  component  sources  will  cause 
reductions  in  the  effective  system  bandwidth  relative  to 
those  in  flight.  If  the  bandwidth  changes  occur  in  a 
rating  sensitive  region,  the  simulator  will  be  more 
poorly  rated  than  flight  for  this  reason  alone". 
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4.6  Criteria  of  Acceptability 
Criteria  of  acceptability  depend  strongly  on  the 
application  of  the  simulator,  e.g.  if  it  is  used  as  a 
training  device,  as  a  handling  qualities  development  tool 
or  as  a  testing  tool  for  real  hardware  system  validation 
or  acceptance. 

For  piloted  simulators  the  acceptability  criteria  require 
that 


by  expert  judgement  (Fig  3).  Typical  accepted 
deviations  arc  about  1-10  %  as  a  function  of  the  variable 
and  the  application  19 . 

A  pre-requisite  for  using  this  technique  is  that  the 
measurement  errors  and  inaccuracies  in  the  measured 
data  are  known  and  extracted  before  the  comparison  is 
made.  Due  to  the  'open-loop'  nature  of  this  method,  any 
incorrect  initial  values  of  the  model  will  lead  over  time 


Task  performance 

Handling  qualities  and  pilot 
workload,  and 

Control  activity 

are  all  satisfactory  representations  of 
the  same  characteristics  on  the  aircraft 
in  flight.  Unfortunately  there  are  no 
firm  guide-lines  on  what  constitutes  a 
satisfactory  representation.  The  best 
guidance  is  given  in  the  documentation 
that  supports  handling  qualities 
criteria,  and  this  needs  to  be  combined 
with  sound  judgement  from  experi¬ 
enced  simulation  engineers,  aircraft 
designers  and  pilots. 

Handling  qualities  criteria  describe  the 
influence  of  system  parameters  on 
pilot  assessed  handling  qualities. 
These  data  are  given  in  the  MIL-F- 
87854  and  the  MIL-STD-1797.  and  the 
related  background  information3.  By 
using  identical  methods  to  extract 
handling  qualities  parameters  from  the 
simulator,  it  can  formally  be  shown 
that  the  simulator  fulfils  the  required 
handling  quality  levels.  All  these 
methods  cannot  replace  direct  pilot 
assessment  to  ensure  that  the  behav¬ 
iour  of  the  simulator  corresponds  to 
that  of  the  aircraft.  This  is  especially 
true  if  motion  and  visual  cues  are 
important  features  of  the  simulation 
task. 

4.7  Validation  methods 

4.7.1  Time  domain 

The  technique  most  widely  used  for 
simulation  validation  consists  of 
matching  measured  time  response  data 
from  flight  tests  with  the  simulated 
response  by  using  identical  control 
inputs.  Typical  traces  from  flight  are 
seen  on  Fig  9.  These  are  compared 
with  responses  from  the  simulator  on 
Fig  10.  Tire  comparison  is  made  by 
cross-plotting  measured  and  simulated 
data  or  by  overlaying  the  plots  to 
identify  the  agreement  and  the 
differences.  The  allowed  error  is  given 
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Fig  9:  Example  of  Aircraft  response  data 
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to  large  discrepancies.  To  alleviate  these  difficulties,  so 
called  "closed-loop"  testing  methods  have  been  devel¬ 
oped20’21  so  that  incorrect  initialisation  can  be  identified 
and  corrected.  Nevertheless,  if  discrepancies  remain,  the 
main  problem  is  to  identify  which  part  or  which 
parameter  in  the  model  has  to  be  adjusted,  without  the 
risk  of  changing  other  dynamic  characteristics  of  the 
model. 

4.7.2  Frequency  domain 

For  simulation  validation,  the  comparison  with  the  real 
world  data  can  also  be  applied  in  the  frequency  domain 
by  comparing  frequency  response  data  (e.g.  Bode  plots). 
These  data  are  often  available,  particularly  for  actuation 
systems. 

Frequency  domain  methods  require  the  excitation  of 
both  the  aircraft  or  subsystems  and  the  simulator  by 
sinusoidal  inputs  at  different  frequencies,  or  by  continu¬ 
ously  increasing  frequencies  (sweeps).  The  frequency 
response  is  calculated  either  by  special  frequency 
analysis  equipment  or  by  using  Fast  Fourier  Transform 
methods.  The  resulting  Bode  plots  of  both  systems  are 
overlaid  and  compared  visually.  Any  unacceptable 
differences  in  the  frequency  range  of  interest  can  then 
be  noted. 

Here  again,  the  problem  arises  of  how  the  deviations 
between  the  two  curves  have  to  be  weighted,  in  order  to 
estimate  the  influence  on  handling  qualities.  As  a  guide, 
frequency  response  boundaries  of  deviations  which  are 
unnoticeable  to  the  pilot  as  far  as  handling  qualities  are 
concerned  are  given  in  the  MIL-F-8785  Standard 
Handbook5.  However,  this  information  is  only  available 
for  the  pitch  axis  response. 

In  addition  to  frequency  response  comparison,  the 
validation  can  be  made  by  using  the  equivalent  system 
approach  to  calculate  the  transfer  function  of  the 
equivalent  low  order  system,  and  then  comparing  the 
estimated  transfer  function  parameters  or  the  calculated 
frequency  response.  In  order  to  improve  the  matching 
process  or  to  minimise  the  difference  between  the 
frequency  responses,  automatic  parameter  identification 
methods  for  the  frequency  domain  are  available  22 

The  main  advantage  of  using  frequency  response  data 
for  simulation  validation  is  that  both  the  phase  and  high 
frequency  characteristics  which  are  very  important  for 
closed  loop  stability  (e  g.  PlO-problems)  can  easily  be 
compared.  For  instance,  frequency  response  comparison 
is  very  useful  in  motion  system  validation,  to  evaluate 
acceleration  response  and  wash-out  characteristics. 
Compared  to  the  time  domain  methods,  the  frequency 
domain  methods  require  more  equipment  (hardware  and 
software)  for  system  excitation  and  data  analysis. 

4.7.3  Hardware-in-the-loop 

Typical  examples  of  hardware-in-the-loop  simulation 
are  the  use  of  a  fully  representative  simulation  cockpit 
using  aircraft  hardware  to  provide  the  pilot  with  a  real 
world  environment,  or  the  connection  of  an  "iron  bird" 
to  the  simulation,  on  which  the  aircraft's  hydraulic  and 


electrical  systems,  including  loaded  actuators,  are 
implemented.  Hardware-in-the-loop  is  important  if  high 
realism  is  required,  and  if  the  behaviour  of  the 
subsystem  is  too  complex  to  be  simulated  under  real 
time  conditions.  In  particular,  for  the  simulation  of  the 
new  generation  of  aircraft  with  multiple  redundant  flv- 
by-wire  flight  control  systems,  real  flight  control 
computer  hardware  is  required  to  be  able  to  represent  all 
functions,  modes  and  the  failure  behaviour.  This  is 
mandatory  if  the  flight  control  system  is  to  be  validated 
or  certified  in  the  simulator. 

The  big  advantage  in  using  real  hardware  is  that  no 
effort  is  needed  for  the  validation  of  the  functions  which 
are  represented  by  the  subsystem.  But  if  simulation  is 
used  to  validate  or  certify  hardware  subsystems,  it  must 
be  proved  that  the  data  communication  between 
subsystem  and  simulation  (e.g.  by  bus  system  interfaces 
like  ARINC  429,  MIL-BUS  1553B)  is  identical  to  the 
real  world.  Furthermore,  the  real-time  aircraft  simula¬ 
tion  data  must  be  synchronised  with  the  hardware  and 
calculated  at  the  same  or  a  multiple  of  the  frame  rate  of 
the  real-time  process  of  the  subsystem. 

4.8  Subjective  Assessment 
The  final  stage  of  the  validation  process  is  to  introduce 
a  pilot  into  the  loop,  to  fly  the  task  defined  at  the  outset. 
Obviously,  the  pilot  must  be  familiar  with  flying  the 
task  on  the  actual  aircraft.  It  is  also  important  that  the 
pilot  is  subjected  to  relatively  short  testing  sessions  with 
actual  flying  interleaved.  This  reduces  the  problem  of 
the  pilot  adapting  to  the  simulator  and  any  deficiencies 
it  may  have.  Reference  15  contains  a  rating  scale  for 
simulator  validity  which  has  been  slightly  modified  to 
make  it  independent  of  flying  task  and  is  shown  in  the 
Table  on  the  next  page.  This  is  a  useful  addition  to  the 
use  of  Handling  Qualities  and  Workload  rating  scales 
such  as  the  Cooper-Harper  Handling  Qualities23  and  the 
Bedford  Pilot  Workload24  scales. 

As  well  as  seeking  pilot  opinion  during  an  assessment, 
simulator  fidelity  can  also  be  verified  by  observing  pilot 
behaviour  during  the  task.  Monitoring  the  pilot's 
activities  and  comparing  them  with  flight  activities 
whilst  performing  a  similar  task  can  either  contribute  to 
the  validation  of  a  simulator  or  point  to  areas  where  the 
simulator  might  be  deficient.  Confidence  in  the  simula¬ 
tor  will  be  increased  if  there  are  no  significant  differ¬ 
ences  observed  in  control  activity  or  instrument 
scanning  strategy.  If  it  is  considered  that  these  activities 
differ  significantly  from  flight  experience,  then  the 
reasons  for  these  differences  should  be  established,  so 
that  no  doubt  is  cast  on  the  use  of  the  simulator  for 
acceptance  of  the  aircraft. 

If,  at  the  pilot  assessment  phase,  it  is  concluded  that  the 
simulator  replicates  the  flying  task  in  performance, 
handling  and  workload,  and  in  control  activity,  then  the 
simulator  has  been  validated  overall  for  this  chosen 
task.  However,  should  the  subjective  assessment  high¬ 
light  deficiencies,  some  retracing  through  the  process  is 
necessary'. 
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Category 

Rating 

Adjective 

Description 

Satisfactory 
representation 
of  actual 
vehicle 

1 

Excellent 

Virtually  no  discrepancies;  simulator  reproduces  actual 
vehicle  characteristics  to  the  best  of  my  memory.  Simulator 
results  directly  applicable  to  actual  vehicle  with  high  level  of 
confidence. 

2 

Good 

Very  minor  discrepancies.  The  simulator  comes  close  to 
duplicating  actual  vehicle  characteristics,  simulator  results  in 
most  areas  would  be  applicable  to  the  actual  vehicle  with 
confidence. 

3 

Fair 

Simulator  is  representative  of  actual  vehicle.  Some  minor  dis¬ 
crepancies  are  noticeable,  but  not  distracting  enough  to  mask 
primary  characteristics.  Simulator  trend  could  be  applied  to 
actual  vehicle. 

Unsatisfactory 
representation 
of  actual 
vehicle 

4 

Fair 

Simulator  needs  work.  It  has  many  minor  discrepancies  which 
are  annoying.  Simulator  would  need  some  improvement 
before  applying  results  directly  to  actual  vehicle,  but  is  useful 
for  general  handling  qualities  investigations  for  this  class  of 
aircraft. 

5 

Bad 

Simulator  not  representative.  Discrepancies  exist  which 
prevent  actual  vehicle  characteristics  from  being  recognised. 
Results  obtained  here  should  be  considered  as  unreliable. 

6 

Very  Bad 

Possible  simulator  malfunction.  Wrong  sign.  Inoperative 
control(s),  other  gross  discrepancies  prevent  comparison  from 
even  being  attempted.  No  data. 

4.9  Deficiencies 

Simulator  cueing  is  inevitably  deficient  to  some  degree 
and  it  is  reasonable  initially  to  look  in  this  area  for  the 
cause  of  the  deficiency.  Motion  cues  can  never  fully 
represent  those  of  the  aircraft,  visual  cues  may  be 
restricted  in  terms  of  field-of-view  or  scene  detail  and 
both  suffer  from  transport  delays.  Simulator  cues  should 
be  completely  eliminated  from  the  investigation  as  the 
cause  of  any  deficiency  before  looking  elsewhere  for  the 
cause.  Not  only  should  the  investigation  consider  the 
validity  of  the  cues  provided  by  the  simulator  but  also 
any  cues  which  may  be  missing.  This  would  imply  an 
error  at  the  stage  when  the  necessary  cues  for  the  task 
were  defined.  In  the  worst  instances  this  could  be  as 
dramatic  as  concluding  that  a  motion  system  is  needed 
for  a  simulator  which  was  originally  considered  not  to 
require  one. 

Assuming  that  cues  have  been  eliminated  from  the 
investigation,  the  next  point  to  consider  is  whether  any 
simplifying  system  assumptions  made  at  the  outset  were 
appropriate.  Subtleties  of  handling  qualities  may  be 
missing  as  a  result  of  a  modelling  simplification  thought 
to  be  justified  at  the  time. 

5.  CERTIFICATION  ISSUES 

5.1  Scope  for  Simulation  in  Civil  Aircraft 
Certification 

As  discussed  in  Section  2.1.  Airworthiness 
Requirements  for  civil  aircraft  do  not  specifically  state 
that  flight  simulation  is  an  acceptable  means  of 


demonstrating  compliance,  nor  do  they  exclude  such 
testing.  Paragraph  25.21  of  Reference  1,  relating  to 
"Proof  of  Compliance",  states 

"(a)  Each  requirement  of  this  sub-part  must  be  met 
at  each  appropriate  combination  of  weight  and 
centre  of  gravity  within  the  range  of  loading 
conditions  for  which  certification  is  requested. 
This  must  be  shown  by- 

IT)  tests  upon  an  aeroplane  of  the  type  for 
which  the  certification  is  requested,  or  by 
calculations  based  on,  and  equal  in 
accuracy  to,  the  results  of  testing:  and 

(2)  systematic  investigation  of  each  probable 
combination  of  weight  and  centre  of 
gravity,  if  compliance  cannot  be 
reasonably  inferred  from  combinations 
investigated." 

It  can  be  read  from  this  paragraph  that  simulation  is 
acceptable  if  it  produces  results  equal  in  accuracy  to 
those  from  flight  testing.  This  interpretation  is  now 
widely  accepted  by  both  manufacturers  and  certification 
authorities,  as  described  in  Section  2.1.  The  division  of 
certification  between  actual  flight  testing  and  simulation 
comes  from  a  case  by  case  negotiation  between  the  two 
parties.  To  justify  this  approach,  the  US  Department  of 
Transport  (FA A  Systems  Development  and  Research 
Service)  sponsored  a  study  of  the  benefits  to  be  gained 
from  the  use  of  simulation  in  the  certification  process23. 
The  study,  co-ordinated  by  the  Lockheed-Califomia 
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Company,  surveyed  experience  and  recommendations 
from  Industry,  based  on  an  FAA  Advisory  Circular26, 
taking  account  of  the  economic  and  technical  consid¬ 
erations. 

Two  categories  of  tests  were  identified  as  having  poten¬ 
tial  for  the  use  or  expanded  use  in  simulation.  The  first 
category,  associated  with  Sub-Part  B  of  Reference  1,  is, 
in  the  main,  related  to  performance.  The  tests  are 

25.105  Takeoff 
25. 107  Takeoff  speeds 
25. 109  Accelerate-stop  distance 
25.111  Takeoff  path 
25.113  T akeoff  distance  and  takeoff  run 
25. 1 1 5  Takeoff  flight  path 
25.117  Climb:  general 
25.119  Landing  climb :  all  -engine-operating 
25.121  Climb:  one-engine-inoperative 
25. 123  En-route  flight  paths 
25.125  Landing 
25.149  Minimum  control  speed 
25. 173  Static  longitudinal  stability 
25. 175  Demonstration  of  static  longitudinal 
stability 

25. 177  Static  directional  and  lateral  stability 
25. 179  Demonstration  of  static  directional  and 
lateral  stability 

25.181  Dynamic  longitudinal,  directional,  and 
lateral  stability 

25.253  High  speed  characteristics 

Excluded  from  these  tests  are  those  relating  to  stalling 
and  stall  warning. 

The  second  category,  associated  with  Sub-Part  F  of 
Reference  1,  is  related  to  equipment.  The  tests  are 

25.1301  Function  and  installation 

25. 1303  Flight  and  navigation  instruments 

25.1309  Equipment  systems  and  installations 

25.1321  Instruments-  Arrangement  and  visibility 

25.1323  Airspeed  indicating  system 

25.1325  Static  pressure  systems 

25.1327  Magnetic  direction  indicator 

25.1329  Automatic  pilot  system 

25. 133 1  Instruments  using  a  power  supply 

25.1419  Ice  protection 

25.1431  Electronic  equipment 

25.1457  Cockpit  voice  recorders 

25.1459  Flight  recorders 

Reference  25  also  recommends  that  Sub-Part  D  tests  of 
Reference  1.  relating  to  flight  control  and  stability 
augmentation  (25.671,  25.672),  are  suitable  candidates 
for  the  expansion  of  simulator  usage. 

Additionally,  the  study  reports  a  clear  preference  from 
Industry  to  negotiate  the  use  of  simulation  in  certifica¬ 
tion  on  an  "individual  case"  basis.  The  rationale  behind 
this  recommendation  is  that  the  question  of  what  consti¬ 
tutes  an  acceptable  simulator  is  not  addressed  in  the 
Regulatory  Documents.  The  Industry  preferred  instead 


"guide-lines  concerning  simulator  conformity  and/or 
validation,  rather  than  a  guide  as  to  what  may  or  may 
not  be  simulated." 

In  1983/84.  the  Swedish  Aviation  Authority 
(Luftfartwerket)  sponsored  a  study  of  the  "simulators  for 
certification"  issue.  The  areas  addressed  were 

specific  tests  suitable  for  clearance  on  simula¬ 
tors 

likely  benefits,  including  cost  savings 

required  standards  of  simulation. 

The  initial  study27  was  followed  by  a  series  of  trials  on 
the  SF-340  Development  Simulator  at  Saab,  Linkoping. 
Test  schedules  from  the  SF-340  Flight  Certification 
Programme  were  repeated  on  the  simulator,  using  the 
same  techniques  and  test  pilots  as  used  in  Certification. 
52  test  points  relating  to  performance/flying  qualities 
were  flown.  It  was  concluded  that 

"the  recordings  obtained  from  the  simulator  tie  up 
closely  with  those  obtained  from  the  equivalent 
flight  tests.  Also,  in  most  cases,  the  pilots  rated  the 
task  difficulty  as  similar  to  that  experienced  in 
flight.  Two  test  conditions  were  considered  by  the 
pilots  as  being  more  difficult  than  flight.  The  first 
was  lateral  trimming,  to  measure  static  stability. 
The  second  was  flying  close  to  the  stall." 

Further,  it  was  recommended  that 

"the  areas  where  simulators  should  be  considered 
for  Certification  are:  Avionics,  Failure  Cases, 
Performance  (take-off  and  landing),  and  Handling 
Qualities  -  (especially  when  airplanes  are  modi¬ 
fied)." 

In  the  last  ten  years,  improvements  in  simulator  tech¬ 
nology  have  enhanced  the  accuracy  and  realism  of 
simulators,  and  have  extended  the  areas  where  valid 
ground  testing  is  possible  At  the  same  time,  extensive 
use  of  avionics  in  civil  aircraft  has  added  to  the  certifi¬ 
cation  task.  Much  of  this  additional  load  is  suitable  for 
clearance  on  simulators. 

5.2  Scope  for  Simulation  in  Military  Aircraft 
Clearance 

Although  there  is  a  greater  variation  between  nations  in 
dieir  methods  for  the  specification  and  clearance  of 
military'  aircraft,  there  is  usually  a  clear  indication  to 
manufacturers  that  simulation  plays  a  part  in 
certification.  Taking  Reference  4  as  an  example. 
Chapter  4  states 

"4.1  Compliance  Demonstration.  Compliance  with 
all  requirements  of  section  3  shall  be  demonstrated 
through  analysis.  In  addition,  compliance  with 
many  of  the  requirements  will  be  demonstrated  by 
simulation,  flight  test,  or  both.  The  methods  for 
demonstrating  compliance  shall  be  established  by 
agreement  between  the  procuring  activity  and  the 
contractor.  Representative  flight  conditions. 
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configurations,  external  store  complements, 
loading,  etc.,  shall  be  determined  for  detailed 
investigations  in  order  to  restrict  the  number  of 
design  and  test  conditions.  The  selected  design 
points  must  be  sufficient  to  allow  accurate  extrapo¬ 
lation  to  the  other  conditions  at  which  the  require¬ 
ments  apply. 

Table  XVIII  specifies  general  guide-lines  but  the 
peculiarities  of  the  specific  aeroplane  design  may 
require  additional  or  alternate  test  conditions.  The 
required  failure  analyses  shall  be  thorough,  except¬ 
ing  only  approved  Special  Failure  States." 

(Table  XVIII  has  the  title  "Design  and  Test  Condition 
Guide-lines",  and  relates  the  requirements  to  conditions 
(load  factor,  altitude,  speed,  and  Phase  of  Flight)  to  be 
tested.) 

"4.1.2  Simulation.  The  danger,  extent  or  difficulty 
of  flight  testing  may  dictate  simulation  rather  than 
flight  test  to  evaluate  some  conditions  and  events, 
such  as  the  influence  of  Severe  disturbances, 
events  close  to  the  ground  (except  3. 2. 3. 4  shall  be 
demonstrated  in  flight),  combined  Failure  States 
and  disturbances,  etc.  In  addition,  by  agreement 
with  the  procuring  activity,  piloted  simulation  shall 
be  performed  before  the  first  flight  of  a  new 
airplane  design  in  order  to  demonstrate  the  suit¬ 
ability  of  the  handling  qualities,  and  also  to 
demonstrate  compliance  with  qualitative  require¬ 
ments  in  atmospheric  disturbances  and  in  the 
critical  conditions  identified  in  4.1.11.  (oj  Ref. 4) 
Where  simulation  is  the  ultimate  method  of 
demonstrating  compliance  for  a  requirement,  the 
simulation  model  shall  be  validated  with  flight  test 
data  and  approved  by  the  procuring  activity." 

(paragraph  3. 2. 3. 4  relates  to  longitudinal  control  in 
landing) 

The  difficulty  of  fight  testing  all  combinations  of  Table 
XVIII  is  further  acknowledged  in  the  paragraph  quoted 
below: 

"4.1.3  Flight  Test  Demonstration.  The  required 
flight  tests  will  be  defined  by  operational, 
technical  and  safety  considerations  as  decided 
jointly  by  the  procuring  activity,  the  test  agency 
and  the  contractor  using  results  from  4.1.1  and 
4.1.2  (of  Ref. 4).  It  is  not  expected  that  flight 
demonstration  of  the  requirements  in  Moderate  and 
Severe  disturbances  will  be  done  unless  required 
by  the  airplane  mission.  Some  flights  can  be 
expected  to  encounter  actual  disturbances;  then  the 
qualitative  requirements  would  apply  if  the  distur¬ 
bance  intensity  could  be  categorised." 

The  UK  Flying  Qualities  Requirements28  are  divided 
into  two  parts;  mandatory  requirements,  which  tend  to 
be  qualitative,  and  advisory  information,  in  Leaflets 
which  contain  numerical  values.  The  intention  is  again 


to  allow  negotiation  between  the  vendor  and  the 
customer.  Leaflet  600/1  states: 

"1.9.3  Compliance  with  some  requirements  cannot 
readily  be  determined  quantitatively  by  flight 
testing;  for  example,  some  dynamic  stability' 
requirements,  and  requirements  related  to  theoreti¬ 
cal  turbulence  models.  In  these  cases,  compliance 
can  be  shown  by  theoretical  calculation  or  simula¬ 
tion,  by  agreement  with  the  Aeroplane  Project 
Director,  provided  that  the  data  used  is  derived  as 
far  as  possible  from  flight  testing  and  provided  that 
some  back-up  qualitative  flying  is  done;  for 
example,  some  flying  must  be  done  in  real  turbu¬ 
lence." 

Certification  of  U.S.  naval  aircraft  involves  the  granting 
of  flight  clearance  and  the  approval  of  all  required 
documentation  after  review  by  the  appropriate  authority. 
In  the  case  of  the  U.S.  Navy,  this  authority  resides  at  the 
Naval  Air  Systems  Command  (NAVAIRSYSCOM). 
The  Air  Vehicle  Division  (AIR-530)  has  final  respon¬ 
sibility  for  the  aerodynamics,  control  system,  material 
and  structural  integrity  of  the  aircraft  and  for  determi¬ 
nation  of  critical  conditions  of  flight. 

The  Requirements  for  flight  clearance  of  US  Navy 
aircraft,  in  addition  to  documentation  and  analysis,  call 
for  the  ground  testing  of 

(1)  aircraft  and  ship  compatibility 

(2)  store  separation 

(3)  ground  vibration 

(4)  electromagnetic  effects 

(5)  wind  tunnel  models 

(6)  vibration  and  acoustic  fatigue 

(7)  static  and  shock 

(8)  aircrew  restrictive  effects  per  anthropometric 
accommodation  instructions 

(9)  man-mounted  equipment  compatibility 

(10)  escape  system  compatibility 

and  the  flight  testing  of 

(1)  captive  carriage  of  stores 

(2)  separation  of  stores 

(3)  carrier  suitability 

(4)  flutter  and  divergence 

(5)  acoustic  and  vibration  environment 

(6)  performance 

(7)  handling  qualities 

(8)  loads  and  stress 

(9)  engine,  transmission  and  cross  shaft  perform¬ 
ance 

(10) aircrew  equipment  interoperability 

Information  required  to  support  flight  restrictions, 
envelopes,  and  limits  of  safe  operation,  as  well  as 
operational  suitability  and  supportability  is  derived  from 
extensive  tests,  inspections,  trial  and  evaluation 
programs.  Tire  sources  for  this  data  include: 

(1)  contractor  analyses  and  ground  tests 

(2)  contractor  demonstration  tests 
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(3)  development  test  and  evaluations 

(4)  Board  of  Inspection  and  Survey  (INSURV) 
aircraft  trials 

(5)  technical  evaluations 

(6)  operational  evaluation  (OPEVAL) 

Items  (1)  and  (3)  arc  conducted  by  contractor  test 
engineers  and  test  pilots.  Items  (4)  and  (5)  involve 
contractor  and  Navy  test  personnel  and  Item  (6)  is 
purely  Navy  personnel.  The  complete  testing,  certifica¬ 
tion.  and  OPEVAL  process  often  takes  years  to 
accomplish.  Flight  testing  follows  a  prescribed  format, 
each  level  opening  a  larger  operational  envelope.  A 
typical  sequence  as  followed  by  the  F-14  and  F/A-18 
aircraft  is: 

(1)  ground  check/fault  testing 

(2)  take  off  and  landing 

(3)  low  speed  performance/manoeuvrability 

(4)  high  speed  performance/manoeuvrability 

(5)  air  refuelling 

(6)  stores/weapons  carriage  and  separation 

(7)  aircraft  carrier  suitability 

(8)  high  angle  of  attack  testing 

(9)  spin/departure  testing 

5.3  Economics 

The  case  for  greater  use  of  simulation  in  aircraft  certifi¬ 
cation  rests  to  a  large  extent  on  the  economies  that 
might  ensue.  The  relative  costs  of  operating  develop¬ 
ment  aircraft,  and  performing  equivalent  tests  on  simu¬ 
lators.  are  needed  to  support  the  case. 

Many  studies  have  been  carried  out  to  compare  the 
relative  costs  of  flying  and  ground-based  simulation 
when  used  for  crew  training.  The  costs  that  emerge  van,’ 
with  the  size  and  complexity  of  the  aircraft,  and  the 
nature  of  the  training.  Also,  the  basis  of  the  estimates 
(whether  they  represent  only  the  direct  operating  costs, 
or  include  factors  such  as  aircraft  or  simulator  deprecia¬ 
tion)  introduces  uncertainties  into  the  comparison. 
Typically,  aircraft  operating  costs  are  in  the  range 
$2,000  -  $20,000  per  hour.  At  the  top  end  of  the  range, 
the  ratio  of  aircraft  to  simulator  cost  is  high  -  30:1  is 
typical  for  aircraft  like  the  Boeing  747.  Similar  ratios, 
or  even  higher,  apply  to  operational  training  for 
advanced  military  aircraft.  The  ratio  reduces  to  8:1  for  a 
small  jet  trainer. 

Equivalent  figures  relating  to  development  flying  are 
not  readily  available,  partly  because  the  basis  of  the 
estimate  (what  overhead  costs  to  include,  and  which 
tests  to  include)  is  arbitrary.  Reference  25,  however, 
does  contain  costs,  and  overcomes  the  difficulty  of  what 
overhead  costs  to  include  by 

"assuming  that 

the  cost  of  the  flight  test  aircraft  would  not  be 
considered 

the  cost  of  any  simulator  or  capital  equipment 
used  w'ould  be  considered  as  part  of  the  aircraft 
development  costs. 


Items  included  as  flight  test  costs  were  fuel, 
landing  fees,  insurance,  crew',  data  analysis,  and 
reporting.  Items  included  in  simulator  costs  were 
personnel,  data  analysis,  and  reporting." 

On  this  basis,  the  cost  of  a  flight  test  hour  for  the 
Lockheed  L  1011  (in  1976)  was  estimated  at  $10,000, 
and  the  cost  of  a  simulation  test  hour  at  $1,000.  Rather 
surprisingly,  the  advantage  in  favour  of  the  simulator 
was  reduced,  because  it  was  found  that  although  the 
tests  in  the  simulator  took  the  same  time  to  perform  as 
the  tests  in  the  air,  the  time  in  the  simulator  was  longer 

"because  of  test  repetition  and/or  deviation  from 
the  test  plan  as  requested  by  FAA  and/or  Company 
test  pilots.  Typically  a  single  FAA  test  pilot  is 
aboard  the  flight  test  demonstration,  while  a  great 
many  more  may  be  present  for  a  simulator  test. 
The  speed  at  which  changes  in  configuration  may 
be  made  on  a  simulator  obviously  influences  the 
requests  for  repeated  or  special  tests  by  witnesses." 

According  to  Reference  25,  the  Certification  Flight 
Tests  for  the  L1011  which  were  suitable  for  the  use  or 
expanded  use  of  simulators  required  approximately  240 
flight  test  hours  ($2.4M)  to  demonstrate  compliance. 

Typically,  the  certification  programme  for  a  new' 
transport  aircraft  takes  11-12  months  from  first  flight, 
and  involves  1200  flying  hours.  The  qualification  for 
service  use  of  military  aircraft  is  much  longer,  and  is  a 
much  more  expensive  process.  The  clearance  from  first 
flight  to  service  use  for  an  advanced  strike  aircraft  can 
be  as  long  as  four  years,  and  can  involve  up  to  nine 
development  aircraft.  Between  4000  and  8000  flying 
hours  are  needed,  and  the  cost  per  hour  is  unlikely  to  be 
less  than  $20,000.  Clearing  weapon  systems,  and 
proving  operational  effectiveness,  particularly  in  low' 
level  flight,  are  particularly  expensive  because  of  the 
additional  costs  of  special  facilities,  such  as  test  areas, 
ranges  and  ordnance. 

At  the  same  time,  the  manufacturers  of  such  aircraft  will 
have  invested  heavily  in  ground  testing,  and  in  particu¬ 
lar,  in  flight  simulation.  It  is  therefore  likely  that  the 
standard  of  simulation  used  for  aircraft  development  can 
be  used  with  little  additional  cost  for  flight  test  support 
and  aircraft  clearance.  The  longer  timescales  of  the 
flight  test  programme  also  ensure  that  the  aerodynamic 
and  systems  modelling  is  supported  by  early  flight  test 
results. 

5.4  Simulator  Qualification 

When  an  aircraft  is  offered  by  a  Manufacturer  to  a 
Certification/Acceptance  Authority  for  approval,  the 
standard  of  the  aircraft  must  conform  to  the  standard  of 
the  aircraft  which  will  enter  service.  Correspondingly, 
the  standard  of  a  simulator  being  offered  to  the 
Authority  as  an  alternative  means  of  assessment  must  be 
shown  to  be  representative  of  the  aircraft  in  the  respects 
which  relate  to  the  intended  tests.  The  difficulty  of 
legislating  the  required  standard  presents  a  major 
obstacle  to  formalising  the  use  of  simulators  for  certifi- 
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cation  testing.  Sections  3  and  4  of  this  present  report 
present  tire  main  technical  issues  associated  with  simu¬ 
lator  validation. 

In  the  case  of  civil  aircraft  certification,  a  parallel  can 
be  drawn  betw  een  these  issues  and  the  formal  qualifica¬ 
tion  of  a  simulator  for  crew  training.  It  is  worthwhile 
therefore  to  review'  the  process  by  which  a  training 
simulator  is  approved. 

In  the  late  1970s.  tire  US  FAA  introduced  their 
Advanced  Simulator  Plan.  The  approval  requirements 
are  contained  in  Reference  26.  It  is  an  advisory  docu¬ 
ment  which  in  its  introduction  explains  that  it  sets  out 
"one  means  that  would  be  acceptable  to  the 
Administrator  for  the  evaluation  of  airplane  simula¬ 
tors....".  The  Advisory  Circular  specifies  that  an 
Approval  Test  Guide  must  be  developed.  This  approval 
test  guide  is  a  comparison  of  the  simulator  versus  the 
aeroplane  static  and  dynamic  response  characteristics.  It 
is  a  means  that  the  FAA  can  use  to  verify  that  a  simula¬ 
tor  possesses  sufficient  fidelity  when  matched  against 
the  aircraft  to  meet  the  Phase  I.  II  or  III  Standards  of 
training  outlined  in  their  Advisory  Circular. 

The  ATG  not  only  is  required  to  prove  the  performance 
of  the  simulator  against  prescribed  data  but  is  also 
required  to  "describe  clearly  and  distinctly  how  the 
simulator  will  be  set  up  and  operated  for  each  test".  The 
use  of  a  driver  program  to  automatically  accomplish  the 
tests  is  encouraged  but  is  not  mandatory.  Such  a  driven 
ATG  is  known  as  an  Automated  or  Auto-ATG  and 
previously  had  to  be  designed  in  such  a  maimer  that,  at 
any  time  in  the  test,  the  driver  can  be  disconnected  and 
the  manoeuvre  continued  by  the  pilot,  to  completion. 
This  requirement  has  been  dropped  for  simulators 
evaluated  under  FAA  Advisory  Circular  AC120-40B3 
and  thus  back-driving  of  controls  is  a  matter  of  choice. 
The  value  of  an  Auto-ATG  can  only  really  be  appreci¬ 
ated  by  someone  who  has  tried  to  manually  reproduce 
the  control  inputs  contained  in  the  check-out  data  to 
such  a  level  as  to  achieve  simulator  results  which  match 
the  aircraft  from  which  the  data  was  derived.  The  task 
of  manually  flying  these  tests  is  one  which  takes  a  fair 
amount  of  practice,  not  to  mention  skill. 

In  order  to  produce  an  ATG,  check-out  data  is  required. 
The  source  of  such  data  is  closely  controlled.  Firstly  it 
must  be  flight  test  data,  except  in  some  limited  instances 
which  w'ill  be  covered  shortly.  Unless  otherwise  speci¬ 
fied  the  data  should  reflect  the  aeroplane  performance  at 
normal  operating  weights  and  centres  of  gravity  and 
must  be  traceable  to  a  particular  aircraft,  identified  by 
its  tail  number  or  registration,  on  a  particular  flight.  If 
the  only  test  carried  out  is  one  from  an  extreme  operat¬ 
ing  configuration  it  must  be  balanced  by  a  test 
conducted  at  the  opposite  extreme.  The  only  exceptions 
to  the  requirement  to  use  aircraft  flown  data  are  to  cater 
for  aeroplanes  first  certificated  prior  to  June  1980,  when 
after  reasonable  attempts  to  obtain  such  data  have  failed 
alternative  data  may  be  submitted  to  the  FAA  for 
approval  together  with  an  explanation  of  the  reason  for 


and  the  source  of  the  substitution.  In  the  case  of  an 
aeroplane  not  yet  certificated  some  predicted  data  may 
be  used  after  the  agreement  of  the  FAA  but  this  will, 
usually,  have  to  be  replaced  when  certificated  aircraft 
data  does  become  available.  The  use  of  substitute  data 
has.  for  example,  been  approved  in  the  case  of  the  A320 
aircraft  because  of  its  highly  augmented  flight  controls. 
Flight  manoeuvres  which  needed  to  be  flown  in  order  to 
obtain  some  data  for  simulation  purposes  could  not  be 
obtained  because  the  aircraft's  built  in  protection 
devices  prevented  them  from  being  flown.  Accordingly 
the  Regulatory  bodies  have  agreed  to  the  use  of 
engineering  data  obtained  in  well  defined  conditions. 

Once  Evaluated  and  Approved,  an  operator  may  use  the 
simulator  in  an  approved  training  program  and  take 
advantage  of  the  credits  awarded  to  it  by  virtue  of  its 
Phase  of  Approval.  To  maintain  this  qualification,  the 
simulator  will  be  evaluated  on  a  recurring  basis  using 
the  currently  approved  Master  ATG.  Unless  otherwise 
determined  by  the  FAA,  these  recurrent  evaluations  will 
be  accomplished  by  a  Simulator  Evaluation  Specialist 
every  four  months.  Scheduled  to  last  no  longer  than 
eight  hours  per  inspection,  the  testing  will  comprise  of 
one-third  of  the  validation  tests  in  the  MATG  plus  all 
the  Functional  tests  with  the  aim  of  completing  the 
whole  of  the  MATG  annually. 

In  the  past  two  years,  an  International  Working  Group 
under  the  auspices  of  the  Royal  Aeronautical  Society 
has  met  to  develop  common  standards  for  simulator 
approval,  so  that  expensive  duplication  of  tests  to  satisfy 
tire  particular  requirements  of  National  Authorities  can 
be  avoided.  The  Group's  recommendations29  have  been 
adopted  by  ICAO.  Criteria  are  based  on  the  US  FAA 
requirements,  as  summarised  above,  and  variations  from 
them  which  appear  in  the  requirements  of  other 
Authorities. 

The  recommendations  of  the  International  Working 
Group  provide  a  sound  basis  for  more  detailed 
developments,  which  will  address  the 

type  and  number  of  validation  tests 

flight  conditions  for  each  test 

methods  of  proving  compliance 

objective  and  subjective  testing 

classification  of  simulators  into  levels  of 
complexity. 

This  work  will  be  a  good  basis  for  formulating 
acceptance  procedures  for  simulators  intended  for 
aircraft  acceptance.  It  is  a  daunting  task,  however, 
because  the  standards  required  are  likely  to  be  more 
severe  than  those  for  training  simulators.  If  a  training 
simulator  fails  to  represent  the  aircraft  in  some  respect, 
the  consequence  could  be  a  flaw  in  pilot  training,  which 
might  then  result  in  incorrect  procedures  in  flight.  But  if 
the  certification  simulator  fails  to  represent  the  aircraft, 
the  consequences  could  be  much  more  serious.  An 
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aspect  of  aircraft  behaviour  might  be  wrongly  certified, 
with  direct  implications  for  passenger  safety.  It  is  to  be 
expected,  therefore,  that  the  present  arrangement,  of 
case-by-case  approval,  will  continue  to  apply,  and  that 
the  number  of  cases  will  increase  as  the  capability  of 
modem  simulators  is  recognised. 

Formalising  the  use  of  simulators  to  clear  military 
aircraft  presents  a  different  challenge.  There  is  no 
internationally  recognised  standard  for  military  aircraft 
training  simulators  which  is  equivalent  to  the  one 
discussed  above  for  civil  training.  Consequently,  there  is 
not  an  agreed  basis  for  validation  and  acceptance  testing 
of  military'  simulators.  But  the  drawback  of  legal 
liability  for  passenger  safety,  if  the  acceptance 
procedures  are  inadequate,  does  not  have  the  same 
implications  for  military  aircraft. 

It  has  been  shown  in  Section  5.2  that,  because  of  the 
scale  of  testing  needed  for  advanced  military  aircraft, 
the  case  for  the  use  of  simulation  in  acceptance 
procedures  is  greater  than  in  the  civil  case.  It  has  also 
been  shown  that  the  military  specifications  reflect  this 
need.  There  is  therefore  every  incentive  to  encourage 
the  maximum  use  of  simulators  in  military  aircraft 
clearance.  The  way  ahead  is  to  categorise  the  flight 
testing  of  a  typical  new  aircraft  into  areas  which  could, 
wholly  or  in  part,  be  transferred  to  ground-based 
simulation,  given  a  suitable  simulator. 

The  next  task  would  be  to  consider  the  requirements  for 
a  simulator  for  each  of  these  test  conditions,  in  terms  of 
technical  performance  required  from  each  element.  A 
further  consideration  would  the  provision  of  hard 
evidence  that  such  testing  is  equivalent  (or  better)  than 
conventional  flight  testing.  This  aspect  is  vital,  because 
validation  of  the  simulation  has  a  strong  subjective 
component. 

5.5  Other  Issues 

Special  efforts  should  be  applied  to  advancing  the  use  of 
simulation  in  areas  where  other  means  of  acceptance 
testing  are  inadequate  or  inappropriate  and,  in  particu¬ 
lar,  where  considerations  of  safety,  security  and  practi¬ 
cality  may  preclude  certification/acceptance  based  on 
flight  testing.  Extreme  meteorological  conditions  (e.g. 
turbulence  or  wind  shear)  cannot  be  set  up  on  demand  in 
flight.  Suitable  mathematical  models  of  these 
phenomena  exist  and  simulation  could  replace  flight 
trials  for  this  aspect  of  certification/acceptance  now. 
Trials  staged  overseas  in  locations  prone  to  extreme 
atmospheric  conditions  are  time  consuming  and  expen¬ 
sive.  Provided  adequate  data  can  be  obtained  from 
testing  in  environmental  chambers  etc.,  simulation 
could  also  partially  or  fully  replace  such  trials. 

Testing  of  heavily  degraded  and  limiting  failure  modes 
may  be  unacceptably  hazardous  or  impossible  to  simu¬ 
late  in  flight:  e.g.  complex  failure  modes  including 
mechanical  failures.  Simulation  can  be  applied  directly 
to  this  aspect  of  certification/acceptance  testing 
provided  the  characteristics  of  the  failure  modes  can  be 


adequately  defined  from  rig  or  other  tests.  The 
challenge  here  is  to  identify  all  the  critical  failure 
modes,  and  this  applies  equally  to  simulation  or  flight 
based  certification/acceptance  testing. 

Considerations  of  security,  particularly  of  electromag¬ 
netic  emissions,  cost  and  practicality  will  constrain  or 
preclude  flight  testing  to  demonstrate  operation  in 
complex  operational  environments.  But  extensive  co¬ 
operation  with  friendly  surface  and  airborne  forces  (e.g. 
passing  of  intelligence  and  hand-over  of  targets, 
integration  with  battlefield  management  systems,  and 
effective  operation  during  many-on-many  engagements) 
will  be  essential  capabilities  for  future  combat  aircraft. 
Furthermore,  the  drive  towards  procurement  of 
complete  weapon  systems  for  a  fixed  price  will  place 
increased  emphasis  on  the  need  to  prove  total  system 
operational  effectiveness  as  opposed  to  demonstrating 
platform  performance  characteristics.  Such  requirements 
will  place  heavy  demands  on  the  development  of 
sophisticated  models  of  friendly  and  hostile  weapon 
systems,  and  on  capabilities  to  network  simulators. 
However,  these  efforts  will  be  equally  applicable  to  the 
provision  of  adequate  full  mission  training  simulators 
for  such  aircraft,  and  simulators  with  these  capabilities 
could  also  play  a  significant  role  in  the  development  of 
tactics  and  Operational  Assessment. 

The  strongest  thread  running  through  all  these  require¬ 
ments  is  the  need  for  increased  sophistication,  and 
scope,  of  modelling  and  model  validation.  This  in  turn 
will  require  extensive  data  gathering  and  analysis, 
improved  tools  for  software  development,  and 
improvements  in  computer  hardware  and  software 
efficiency.  Improvements  in  other  fields  of  simulation 
technology  (e.g.  visual  and  motion  cueing)  may  be 
appropriate  for  some  applications  (e.g.  ultra  low  level 
and  NOE  flight  of  highly  agile  aircraft/helicopters  and 
visual  resolution  to  identify  targets  at  extreme  range), 
but  in  general  any  shortcomings  in  the  best  current 
systems  seem  unlikely  to  be  so  severe  as  to  preclude  the 
use  of  simulation  for  the  kinds  of  testing  described 
above.  Nevertheless,  the  progressive  reduction  in  cost 
for  similar  levels  of  capability  must  be  as  welcome  in 
this  application  as  it  is  for  training  simulators. 

6  CONCLUSIONS 

Piloted  simulators  are  already  being  used  as  part  of  the 
certification  or  acceptance  process  for  aircraft  and, 
particularly  their  sub-systems.  However,  there  is 
currently  little,  if  any,  guidance  for  clearance  authorities 
or  manufacturers  to  ensure  that  simulators  are,  and  are 
seen  to  be,  validated  effectively.  Current  practice  relies 
heavily  on  the  subjective  opinion  of  experienced  test 
pilots  and,  whilst  this  is  an  essential  part  of  any  valida¬ 
tion,  it  can  only  be  relied  upon  after  rigorous  verifica¬ 
tion  of  mathematical  models  and  validation  of  the 
behaviour  of  simulation  systems  has  been  completed 
satisfactorily.  An  appreciation  of  the  effects  of  simula¬ 
tion  systems  on  the  responses  of  pilots  when  performing 
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relevant  tasks  are  particularly  important.  It  is  in  these 
systems  that  the  'art'  of  simulation  is  used  to  represent 
the  real  visual  scenes,  motion,  sounds,  and  response  of 
instruments  that  the  pilot  uses  to  identify  his  situation 
and  determine  his  actions.  In  nearly  all  respects  the 
simulated  'world'  cannot  respond  in  exactly  the  same 
way  as  the  real  world. 

A  simulator  will  be  satisfactory  for  clearance  activities 
if  there  is  confidence  that  any  differences  in  simulator 
response  are  so  small  that  pilot  response  and  perform¬ 
ance  in  the  relevant  clearance  task  is  the  same  as  it  will 
be  in  flight.  Thus  a  definition  of  a  valid  simulation  for  a 
given  task  is: 

A  piloted  simulation  will  be  valid  for  a  selected 
task  when  the  representation  by  individual 
components  of  the  simulation  and  by  the 
complete  simulation  meets  both  the  quantita¬ 
tive  requirements  of  the  manufacturer/authority 
and  the  subjective  evaluation  of  pilots  with 
relevant  experience  such  that  a  manufacturer 
will  confidently  present  simulation  as  the  sole 
means  of  demonstrating  compliance  with  the 
Authority's  certification  or  acceptance  require¬ 
ments  for  some  element  of  that  task. 

The  main  involvement  of  simulators  in  certification  at 
present  is  in  the  selection  of  cases  for  flight  demonstra¬ 
tion:  in  providing  opportunities  to  determine  piloting 
procedures  and  to  practice  them  before  flight  tests,  and 
in  demonstrations  for  certifying  authorities.  However, 
there  are  a  growing  number  of  examples  where  simula¬ 
tion  has  been  used  as  the  sole  demonstration  of  particu¬ 
lar  cases  for  flight  clearance.  Examples  begin  with  the 
NASA/USAF  Space  Shuttle,  where  a  normal  flight 
development  programme  was  impractical,  Appendix  B, 
and  more  recently  include  aspects  of  the  Airbus  A320 
and  Fokker  F100  airliners.  (Appendix  A),  where  certain 
less  critical  or  very  low  probability  flight  control  system 
failures  have  only  been  demonstrated  in  piloted  simula¬ 
tion.  This  also  includes  rare  and  extreme  meteorological 
situations  that  are  almost  impossible  to  obtain  or  repre¬ 
sent  in  flight. 

Military'  examples  have  been  less  obvious  in  the  past 
because  it  is  not  practical  to  have  such  well  defined 
certification  procedures  to  deal  with  the  multiplicity  of 
configurations,  roles  and  physical  environments 
encountered  by  a  single  type  of  aircraft.  However, 
simulation  is  used  as  one  means  of  identifying  cases 
which  must  be  demonstrated  in  flight  from  the  very 
wide  range  of  configurations  and  failures  that  must  be 
evaluated.  Thus  piloted,  or  theoretical,  simulation  is 
being  used  implicitly  to  clear  cases  that  are  not  demon¬ 
strated  in  flight. 

More  recent  examples  of  the  use  of  simulation  in  the 
clearance  of  military  aircraft  such  as  the  Tornado. 
AMX,  BAe  EAP,  and  F14A  are  given  in  Appendix  B. 


Essential  stages  in  validating  a  simulator  for  a  clearance 
task  are  shown  in  Fig  2.  There  are  four  primary7 
activities 

•  Model  validation 

•  Cue  validation 

•  Integrated  system  validation 

•  Pilot  in-the-loop  assessment 

Modelling  and  simulation  validation  is  a  mainly 
analytical  activity  and  the  most  important  pre-requisites 
are 

•  Good  quality  sources  of  modelling  data,  e.g. 
wind  tunnel,  flight  test,  etc. 

•  Appropriate  independent  flight  data  to  test 
validity  in  certification  task  areas,  i.e.  not  the 
same  test  data  used  to  derive  the  model. 

In  general,  manufacturers  are  unlikely  to  propose  simu¬ 
lation  for  clearance  demonstration  unless  the  aircraft  has 
good  modelling  data  sources.  Independent  flight  data  for 
validation  is  readily  available  for  civil  airliners  where 
training  simulators  wall  be  an  intrinsic  part  of  any  sales 
of  aircraft  and  standards  exist  defining  simulator 
modelling  and  certification  requirements.  For  military 
aircraft  there  are  no  standards  requiring  flight  test  data 
for  simulator  modelling  or  certification.  It  will  be  much 
less  expensive  to  incorporate  data  collection  for  model¬ 
ling  and  validation  during  the  initial  development  flight 
trials  of  an  aircraft,  when  similar  tests  are  required  for 
development,  than  it  will  be  to  carry  out  a  separate 
series  of  special  tests  at  a  later  stage.  This  requires 
acknowledgement  by  government  aircraft  project  offices 
that  provision  of  simulation  modelling  and  validation 
data  is  part  of  an  aircraft's  development  and  not  a  sepa¬ 
rate  simulation  procurement  activity. 

A  particularly  important  part  of  model  validation  must 
be  testing  of  the  software  for  faults.  Associated  with  this 
is  a  need  for  visible  and  effective  software  configuration 
management  to  ensure  that  any  validated  simulation  is 
associated  with  a  unique  and  properly  safeguarded  set  of 
software,  which  must  include  the  simulation  operating 
software  as  well  as  specific  models.  Faulty  logic  will 
not  necessarily  be  identified  by  matching  responses  to 
tests  at  discrete  conditions,  and  yet  can  be  a  source  of 
significant  illogical  discrepancies. 

Cue  validation  is  primarily  associated  with  the 
physiological  and  cognitive  response  of  pilots  to 
physical  stimuli.  Thus,  although  important  guide-lines 
can.  and  should,  be  provided  to  design  and  assess  the 
probable  capabilities  of  a  simulator  for  given  tasks,  the 
effectiveness  of  the  complete  system  of  motion,  visual, 
tactile  and  aural  cues  can  only  be  evaluated  by  the  final 
activity  of  pilot  assessment.  However  it  is  important  that 
quantitative  measures  of  the  quality,  including 
dynamics,  of  these  cues  are  available  to  avoid  known 
problems  and  to  provide  a  basis  for  analysing  and  curing 
deficiencies  identified  during  pilot  assessments. 
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One  particular  issue  that  frequently  arises  is  the  neces¬ 
sity,  or  otherwise,  of  motion  cueing.  This  is  a  major  part 
of  the  general  issue  of  the  importance  of  dynamic  cues 
that  the  pilot  uses  for  direct  control  of  the  aircraft 
(Section  3.1.1).  Any  decision  not  to  use  motion  cues 
must  be  approached  carefully.  The  following  are 
particular  examples  where  confidence  in  validity  would 
be  significantly  reduced  if  motion  cueing  were  absent: 

•  situations  where  the  aircraft  with  its  rele¬ 
vant  control  system  has  low  or  negative 
stability 

•  situations  where  external  disturbances  or 
the  motion  changes  caused  by  failures  are 
abrupt,  e  g.  engine  failure 

•  aircraft  with  active  control  systems  where 
control  surface  motions  are  not  linearly  or 
simply  related  to  inceptor  (e.g.  control 
stick)  position. 

The  last  case  is  important  because  active  control 
systems  can  produce  unusual  motion  at  the  cockpit 
through  attempts  to  increase  control  effectiveness  or  to 
modify  natural  cross-coupling  from  control  surface 
inputs.  Control  problems  can  arise  from  pilots’  natural 
reactions  to  these  cockpit  motions  that  will  not  be 
present  in  the  simulator  if  there  are  no  motion  cues. 

Evaluation  of  the  integrated  model  and  cueing  systems 
is  used  to  check  the  overall  frequency  responses  and 
end-to-end  time  delays  of  the  system.  These  need  to  be 
satisfactory  before  the  simulator  is  presented  to  a  pilot 
for  the  final  stage  of  validation. 

With  the  pilot  in-the-loop  the  simulator  is  assessed  in 
three  areas 

Task  performance 

Handling  qualities  and  pilot  workload,  and 

Control  activity. 

Only  if  the  simulator  is  a  satisfactory  representation  of 
the  aircraft  in  all  three  areas  can  it  be  accepted  as  valid 
for  clearance  activities. 

For  pilot  assessment  the  most  important  requirements 
are  that  the  pilots  are  very  familiar  with  the  aircraft 
behaviour  in  regimes  similar  to  the  tasks  to  be  demon¬ 
strated.  that  they  are  currently  flying  the  aircraft,  and 
that  they  are  trained  in  the  analysis  of  flight  activities, 
e.g.  a  qualified  test  pilot.  These  requirements  are  par¬ 
ticularly  important  because  it  is  easy  for  any  experi¬ 
enced  pilot  to  learn  to  fly  a  simulator,  but  only  pilots 
with  relevant  experience  and  training  can  assess  the 
validity  of  a  simulator  to  represent  a  particular  aircraft 
and  task.  The  'currently  flying'  requirement  is  particu¬ 
larly  important  because  experienced  pilots  will  quickly 
adapt  to  a  simulator  and  will  find  it  difficult  to  judge  the 
significance  of  apparent  differences  without  trying 
similar  checks  in  flight. 


Throughout  all  these  validation  (and  verification) 
activities  there  is  a  need  for  appropriate  criteria  for 
assessing  the  acceptability  of  the  simulator  or  one  of  its 
components.  For  model  verification  it  is  usual  to  set 
deviation  limits  for  output  variables  of  between  1  and 
10%  depending  on  the  variable  and  the  task.  However, 
there  is  significant  professional  judgement  required  and 
the  process  cannot  be  totally  automated.  For  cueing 
systems  and  overall  pilot  assessment  there  are  no 
accepted  quantitative  criteria.  However,  experienced 
simulation  engineers  have  a  body  of  knowledge  of 
appropriate  'good  practice'  for  simulating  particular 
tasks.  There  is  a  need  to  codify  some  of  this  knowledge 
for  cueing  systems.  This  should  lead  to  the  development 
of  accepted  criteria  and  better  dissemination  of  the 
available  knowledge  to  all  simulator  engineers. 

Only  after  all  the  systems  have  been  accepted  as  suit¬ 
able  for  the  tasks  can  a  pilot  make  an  overall  assess¬ 
ment.  It  is  particularly  important  that  this  assessment  is 
reported  using  rational  and  generally  accepted  rating 
scales  such  as  the  overall  scale  proposed  in  the  Table  of 
Section  4.8  (p.22)  and  other  accepted  rating  scales  for 
particular  parts  of  the  activity,  such  as  the  Cooper- 
Harper  scale23  for  Flying  qualities,  and  the  Bedford 
Pilot  Workload  scale24. 

The  legal  frameworks  for  certification  of  both  military 
and  civil  aircraft  in  Europe  and  N.  America  accept  that 
simulation  may  be  used  for  parts  of  the  process  and  do 
not  stipulate  any  cases  that  may  not  be  cleared  through 
simulation.  However,  each  case  where  a  manufacturer 
proposes  to  use  simulation  will  be  reviewed  by  the 
certifying  authority  and  there  are  no  cases  where  simu¬ 
lation  will  be  automatically  accepted. 

Studies  in  the  USA  (1977)25  and  Sweden  (1983)27  both 
concluded  that  simulation  was  potentially  acceptable  for 
most  areas  of  civil  aircraft  certification  with  the 
probable  exception  of  stall  and  stall  warning.  This 
exception  is  not  surprising  as  modelling  of  this  region  is 
particularly  difficult,  good  motion  cueing  is  essential, 
and  motion  systems  of  that  time  were  unlikely  to 
provide  adequate  quality.  Motion  systems  and  their 
drive  algorithms  have  improved  significantly,  but  the 
modelling  difficulties  still  remain.  Thus  stall  demon¬ 
stration  will  continue  to  be  a  flight  case  for  some 
considerable  time.  The  other  important  point  to  note 
from  the  studies  is  that  the  Industry  prefers  "guide-lines 
concerning  simulator  conformity  and/or  validation, 
rather  than  a  guide  as  to  what  may  or  may  not  be  simu¬ 
lated." 

This  is  particularly  important  as  continuing  develop¬ 
ments  in  simulation  quality  will  bring  an  increasing 
range  of  certification  tasks  within  the  scope  of  simula¬ 
tion,  and  also  the  simulation  systems  of  different  manu¬ 
facturers  will  be  suitable  for  different  ranges  of  tasks 
and  vehicles. 

The  growth  of  complexity  in  aircraft  systems,  military 
roles,  and  certification  requirements  are  all  increasing 


30 


the  total  scope  of  the  certification  process.  The  financial 
savings  available  through  using  simulation  instead  of 
flight  testing  are  considerable.  Then  there  are  tests  in 
severe  weather  or  of  very  rare  major  system  failures  that 
must  be  cleared  bv  simulation  as  they  are  almost 
impossible  to  test  in  flight.  Surv  eys  have  suggested  cost 
ratios  for  operational  training  of  between  8:1  for  a  jet 
trainer  to  30:1  for  a  large  transport  aircraft  per  flight 
hour  in  favour  of  simulation.  In  development  flying  the 
cost  per  flight  hour  will  be  much  higher,  but  there  will 
also  be  a  tendency  to  use  the  flexibility  of  the  simulator 
to  increase  the  range  of  demonstration  required  by  the 
certifying  authority.  How' ever,  the  cost  benefits  of  using 
a  simulator  will  still  be  large.  Thus  it  is  to  be  expected 
that  manufacturers  and  customers  will  wish  to  make  the 
maximum  possible  use  of  simulation  in  the  certification 
process. 

This  means  that  both  manufacturers  and  certifying 
authorities  require  guide-lines  and  procedures  for 
assessing  and  demonstrating  simulation  validity. 

Structured  guide-lines  and  requirements  for  simulation 
quality  in  existence  at  present  are  the  FAA  requirements 
for  civil  flight  training  simulators  and  flight  training 
devices3.  An  important  feature  of  these  regulations  is 
the  requirement  for  generating  an  Approval  Test  Guide 
(ATG).  This  includes  comparisons  of  the  simulator 
responses  with  independent  flight  test  data,  and  must 
"describe  clearly  and  distinctly  how'  the  simulator  will 
be  set  up  and  operated  for  each  test".  This  ATG, 
together  with  an  assessment  by  an  FAA  pilot,  is  the 
means  by  which  training  simulators  gain  approval  (and 
annual  recertification)  to  replace  certain  flight  training 
activities.  The  highest  level  allows  a  pilot  to  convert  to 
a  new7  aircraft  type  without  any  flight  experience  on  the 
new  aircraft  before  his  first  revenue  flight.  This 
requirement  for  a  Test  Guide  is  also  a  key  feature  of  the 
recent  International  Standard  for  the  Qualification  of 
Airplane  Flight  Simulators29  presented  to  the 
International  Civil  Aviation  Organisation  in  1992  by  an 
International  Working  Group  of  the  UK  Royal 
Aeronautical  Society. 

These  procedures  for  civil  training  simulators  provide  a 
sound  basis  for  developing  assessment  methods  to 
qualify  simulators  for  flight  clearance  demonstrations. 
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Appendix  A 


EXAMPLES  OF  THE  USE  OF  SIMULATION  IN 
THE  CERTIFICATION  OF  CIVIL  TRANSPORT 
AIRCRAFT  SYSTEMS 


A.1  USE  OF  SIMULATION  FOR  AIRBUS  A320 
CERTIFICATION 

A320  development  and  certification  have  made  wide 
use  of  simulators.  These  simulators  were  used  for 
various  tasks,  such  as  flight  control  law  definition, 
cockpit  ergonomics  and  man-machine  interface  studies, 
equipment  integration,  initial  training  of  flight  test 
crews,  validation  of  equipment  modifications,  analysis 
of  failure  cases  and  responses  to  atmospheric 
disturbances.  They  have  greatly  enhanced  the 
productivity  and  safety  of  the  flight  tests  and  have 
provided  an  ideal  complementary  test  facility. 

A.1.1  Development  Simulators 

A  specific  tool  was  created  by  Aerospatiale  in  1977,  to 
support  aircraft  Research  and  Development,  named 
EPOPEE  (Etude  Prospective  pour  l'Organisation  d'im 
Poste  d'Equipage  Ergonomique).  The  EPOPEE  simula¬ 
tor,  a  fixed-based  simulator  with  visual  system,  is  still 
used  in  the  area  of  cockpit  ergonomics  and  man- 
machine  interface.  Its  specific  contribution  to  the  A3  20 
included  such  important  items  as  the  selection  of  large 
7.25"  EFIS  displays  with  side-by-side  location  for  the 
Primary  Flight  Displays  and  Navigation  Displays,  the 
design  of  engine  parameter  presentations  on  ECAM 
displays,  sidestick  and  armrest  design  and  determination 
of  the  basic  characteristics  of  the  C*  law  selected  for 
pitch  control. 

Additionally,  five  simulators  were  used  by  Aerospatiale 
for  the  specific  development  of  the  A320  aiqplane.  All 
are  of  the  fixed  base  type.  The  first  one  was  an  A300-B2 
type  simulator,  aimed  at  integrating  and  checking  the 
special  autopilot  computer  modified  to  embody  the 
A3 20  type  control  laws  and  fitted  on  A300  SN3.  These 
flight  test  trials  allowed  a  validation  of  the  control  law 
principles,  especially  in  the  areas  of  flight  envelope 
protection  and  lateral  control  laws.  The  role  of  the 
simulator  was  to  interface  with  the  real  modified 
autopilot  computer  in  addition  to  the  final  tuning  of  the 
control  laws  with  a  pilot  in  the  loop.  It  was  also  used  on 
a  systematic  basis  before  any  new  computer  box  was 
flight  tested. 

A.  1.2  Flight  Control  System  and  Autopilot  Devel¬ 
opment 

Simulator  testing  of  new  experimental  versions  of  a 
flight  control  computer  before  fitting  it  on  actual  aircraft 
has  greatly  increased  both  effectiveness  and  safety  of 
the  flight  tests.  As  soon  as  comprehensive  A320 
aerodynamic  data  was  available  from  wind  tunnel  tests, 
work  on  the  precise  definition  of  the  control  laws,  with 


gains  pertinent  to  A.320  predicted  characteristics,  was 
initiated  with  the  help  of  the  "A3  20  development 
simulator".  This  was  a  fixed-base  simulator  with  a 
visual  system  and  cockpit  arrangement  fully  representa¬ 
tive  of  A320  design  as  w  ell  as,  of  course,  aerodynamic, 
propulsive  and  ground  handling  models.  As  computers 
were  not  yet  available,  flight  control  laws  were  repre¬ 
sented  through  simulator  software. 

It  is  on  this  development  simulator  that  all  the  control 
laws  were  actually  defined  in  a  joint  process  involving 
pilots  and  engineers,  and  then  specified  to  the  computer 
software  teams.  This  phase  also  addressed  the  reconfig¬ 
ured  control  laws  that  have  to  be  activated  in  failure 
cases. 

The  quality  of  the  aerodynamic  model,  and  that  of  the 
general  representation  of  the  whole  development 
simulator,  was  such  that  these  control  laws  were  not 
significantly  affected  during  the  subsequent  phases  of 
aircraft  development  through  to  production  standard. 
Pitch  law  s  w  ere  kept  virtually  unchanged  for  most  of  the 
flight  phases  while  lateral  law  gains  w'ere  tuned  in  a 
reasonable  band.  The  control  law-  architecture  w;as  not 
modified. 

A.  1.3  Flight  Control  System  Clearance 

The  next  stage  was  the  entry  into  service,  about  one 
year  before  first  flight,  of  so-called  "integration" 
simulators  SI,  S2  and  S3  w'hich  in  fact  phased  out  the 
"development"  simulator  from  which  they  mainly  differ 
by  the  use  of  actual  aircraft  "black  boxes". 

All  three  simulators  are  fixed-base,  and  SI  and  S2  share 
a  common  visual  system  that  can  be  alternatively 
connected  to  one  or  the  other.  In  addition,  SI  may  be 
connected  to  the  "Iron  Bird"  with  an  actual  replica  of 
the  aircraft  electrical  and  hydraulic  systems,  as  well  as 
all  the  flight  control  servo  jacks.  S3  is  more  specifically 
dedicated  to  FMS  activities. 

The  first  use  of  the  integration  simulator  was  to  check 
all  aircraft  computers  as  soon  as  the  first  sets  w-ere 
released  by  the  vendors  and  to  integrate  them  in  an 
environment  fully  representative  of  the  actual  aircraft. 
This  was  really  the  first  opportunity  to  comprehensively 
verify  the  computer  interfacing  and  all  logic  and 
monitoring  that  have  to  be  added  to  the  control  laws  to 
make  a  complete  flight  control  computer.  This 
integration  activity  was  not  limited  to  flight  control,  but 
also  was  used  for  the  autopilot,  flight  warning, 
instruments,  slats/flaps  control,  braking  and  steering. 

During  the  year  preceding  first  flight,  future  A320  test 
pilots  W'ere  given  the  opportunity  to  get  accustomed  to 
the  A3  20  handling  characteristics,  so  that  integration 
simulators  also  played  an  effective  training  role.  After 
first  flight,  simulators  were  systematically  used  to  check 
the  performance  of  new'  computer  softw  are  or  hardware 
before  flight  test.  A  comprehensive  simulator  test 
program  was  used  to  specifically  evaluate 
modifications.  This  lead  to  a  more  safe  and  productive 
flight  test  program. 
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A.1.4  Failures 

Auotlier  area  where  the  simulators  have  also  been  used 
extensively  is  for  the  assessment  of  failure  cases,  both  in 
the  development  phase  but  mainly  in  the  certification 
process.  The  simulator  has  proved  to  be  particularly 
suited  to  this  purpose  because: 

1)  It  makes  it  possible  to  simulate  failures  that  are 
very'  difficult  to  obtain  in  flight  (such  as  ADC/IRS 
parameter  drift,  radio-altimeter  failures,  and  mechanical 
failures). 

2)  It  provides  time  to  analyse  the  warnings  and 
procedures,  to  reproduce  the  tests  as  many  times  as 
required,  and  to  cover  a  wider  spectrum  of  flight  condi¬ 
tions. 

3)  It  avoids  exposing  the  aircraft  (and  lives)  to 
high  risk  conditions  such  as  double  failures,  or  very 
remote  failure  states  which  would  be  dangerous  if 
encountered  for  the  first  time  in  flight. 

Finally,  ground  based  simulators  have  also  been  used  to 
reproduce  controlled  meteorological  conditions  that  are 
otherwise  difficult  to  get  from  mother  nature  when 
required.  This  applies  to  the  simulation  of  turbulence, 
associated  or  not  with  failure  cases,  and  to  the  simula¬ 
tion  of  wind  shear. 

A.  1.5  Simulator  Standards 

In  order  to  perform  all  the  above  mentioned  tasks  prop¬ 
erly,  and  get  credit  for  the  use  of  simulators  for 
certification,  a  high  level  of  representation  has  been 
sought.  For  example,  aerodynamic,  engine,  and  ground 
math  models  are  identical  to  these  delivered  to  training 
simulator  manufacturers,  as  they  are  under  the 
responsibility  of  the  same  teams  who  continuously 
updated  the  models  as  soon  as  flight  tests  results  were 
available  for  identification.  The  visual  system  is  a 
commercial,  up  to  date  type  available  for  training 
simulators.  Additionally,  integration  simulators  are 
equipped  with  comprehensive  parameter  recording  and 
display  facilities,  as  well  as  specific  software  or 
hardware  equipment  to  generate  all  the  required 
failures. 

A. 2  USE  OF  SIMULATION  FOR  FOKKER  F100 
CERTIFICATION 

A.2.1  Electronic  Flight  Instrument  System  (EFIS) 
In  the  development  of  the  EFIS  format  for  new  aircraft 
(Fokker  100.  Airbus  A-310,  A-320.  Boeing  747-400), 
simulation  has  played  an  important  role.  Several  simula¬ 
tion  devices,  ranging  from  static  graphics  devices  to 
dynamic  graphics  work  stations  are  being  used  in  this 
process.  As  part  of  an  approved  certification  programme 
to  demonstrate  adequate  and  representative  pilot  expo¬ 
sure  to  the  proposed  Fokker  100  EFIS  Primary  Flight 
Display  format,  quite  extensive  flight  simulation 
programmes  have  been  performed  using  both  the  Fokker 
fixed-base,  single-pilot,  engineering  simulator  with 
actual  avionics  hardware  in  the  loop  and  the  NLR 


general  purpose  moving-base  flight  simulator  equipped 
with  Fokker  100  EFIS  displays. 

A.2.2  Multi-Function  Display  System/Flight  Warn¬ 
ing  System  (MFDS/FWS) 

Many  certification  sessions  have  been  carried  out  on  the 
fixed-base  engineering  simulator  of  Fokker  to  evaluate 
the  functioning  of  the  MFDS/FWS  of  the  Fokker  100.  In 
the  certification  pilot's  report  the  following  was 
concluded:  "The  simulator  evaluation  has  been 

essential  as  a  certification  tool  for  MFDS/FWS.  Ideally, 
much  more  effort  should  be  spent  in  an  investigation 
like  this  on  highly  sophisticated  and  complex  interacting 
avionic  systems.  Tests  on  the  one  hand  helped  to 
validate  various  assumptions  on  paper,  and  on  the  other 
hand  showed  problems  that  were  not  expected  before." 

A.2.3  Autopilot 

The  performance  of  the  Automatic  Flight  Control  and 
Augmentation  System  (AFCAS)  of  the  Fokker  100  has 
been  tested  in  extensive  piloted  and  non-piloted  simula¬ 
tion  exercises.  So-called  "pallet  tests"  have  been 
performed  in  which  a  Fokker  100  prototype  was  used  as 
a  ground-based  simulator  by  interfacing  it  with  a  simu¬ 
lated  set  of  sensors.  The  main  puipose  of  this  "iron  bird 
like"  setup  was  to  efficiently  test  the  behaviour  of  the 
AFCAS  system  under  (risky)  sensor  failure  conditions. 
Of  course,  because  much  information  on  EFIS  is  so 
closely  connected  to  the  autopilot  system,  the  simulator 
tests  for  EFIS  certification  evoked  many  comments 
related  to  autopilot  behaviour  and  thus  can  be 
considered  as  contributions  to  autopilot  certification, 
even  if  it  were  only  for  providing  certification  pilots 
with  svstem  familiarization  time  before  certification 
flights. 

A.2.4  Autoland 

The  proof  that  the  performance  of  an  autoland  system 
meets  the  requirement  of  the  category  for  which  certifi¬ 
cation  is  requested  has  to  be  given  by  a  fast-time 
simulation  in  which  a  large  number  of  factors 
influencing  the  autoland  performance  are  varied  in  a 
random  manner.  However,  before  this  simulation  starts, 
die  simulation  facility  (hardw  are  and  softw  are)  has  to  be 
validated.  In  the  Fokker  100  programme  approximately 
160  actual  automatic  landings  were  performed  for  this 
purpose.  A  programme  of  fast-time  simulations  and 
statistical  comparisons  between  flight  test  results  and 
simulation  results  has  served  to  validate  the  simulation 
program. 

A.2.5  Electronic  and  Hydraulic  Control  Systems 

For  the  Fokker  100,  the  influence  of  failures  in  elec¬ 
tronic  and  hydraulic  systems  on  flying  qualities  always 
has  been  tested  first  in  a  flight  simulator  and  in  many 
cases  (control  surface  jamming),  verification  has  been 
performed  in  real  flight.  However,  low  probability  risky 
occurrences  like  flight  control  jams  or  disconnects, 
thrust  reverser  deployment  in-flight,  stick  pusher  activa¬ 
tion  in  combination  with  wind  shear  are  examples  that 
are  suitable  for  simulation.  An  interesting  example  of 
different  requirements  being  imposed  by  different 
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authorities  is  that,  for  Fokker  100,  the  effect  of  in-flight 
thrust  reverser  deployment  had  to  be  demonstrated  in 
real  flight,  where  for  Airbus  A-320  the  joint 
French/British/German/Dutch  Authorities  have  accepted 
demonstration  by  simulation. 

A.2.6  Flight  Management  System  (FMS) 

This  is  an  example  of  a  system  that  received  extensive 
testing  in  dedicated  system  integration  facilities  and 
piloted  fixed-base  flight  simulator  tests  during  the 
development  phase.  In  this  phase  the  format  of  the 
information  on  EFIS.  the  functioning  of  the  system  and 
the  assessment  of  failures  in  the  system  were 
extensively  assessed  by  certification  pilots.  Final  system 
acceptance  was  performed  in  real  flight.  The  fact  that 
simulation  facilitated  the  familiarisation  of  the 
certification  test  pilots  with  all  aspects  of  the  system 
before  the  actual  certification  flights,  has  reduced  the 
number  of  certification  flight  hours  considerably. 

A.3  SIMULATION  FOR  CERTIFICATION  OF 
THE  ANGLO/FRENCH  CONCORDE 

NASA  simulation  facilities  were  used  to  address  the 
landing  and  take-off  performance  criteria  for  supersonic 
transport  operation.  The  work  specifically  considered 
the  Anglo/French  Concorde,  which  at  that  time  was  well 
on  the  way  to  its  first  flight.  Over  a  three-year  period, 
the  simulation  was  used  by  teams  of  engineers  and 
pilots  from  the  U.S.,  British,  and  French  airworthiness 
authorities,  with  support  of  similar  teams  from  the 
French  and  British  manufacturers.  These  latter  groups 
were  particularly  interested  in  establishing  the  validity 
of  the  simulation,  especially  to  avoid  misrepresentation 
of  the  aircraft's  performance  and  handling  qualities. 
Aerodynamic,  propulsion,  and  control  system  data  of  a 
particularly  comprehensive  nature  were  made  available 
for  the  initial  simulation  and.  as  flight  tests  began,  data 
updating  and  subjective  assessments  by  pilots  who  had 
flown  the  aircraft  combined  to  create  a  high  level  of 
confidence  in  the  validity  of  the  simulation.  It  was 
obvious  that  the  quality  of  the  visual  simulation  system 
and  the  large-amplitude  cockpit  motion  were  important 
to  the  success  of  the  simulation;  however,  it  was 
appreciated  by  all  involved  that  the  success  of  the 
simulator  exercises  was  in  large  part  due  to  the 
continuity  of  individual  participation  of  engineers  and 
pilots  over  nearly  a  three-year  period.  This  fostered 
mutual  understanding  of  the  objectives,  and  the 
development  of  skills  in  the  use  of  simulation.  The 
project  provided  more  strong  evidence  that  pilots  must 
be  given  the  time  to  accommodate  to  the  irreducible 
differences  between  simulator  and  flight,  and  that 
familiarization  with  unusual  aircraft  characteristics  can 
be  expected  to  require  more  time  than  in  flight.  Thus, 
validity  of  the  simulation  is  dependent  upon  the 
evaluation  pilot's  experience  with  the  tool. 


A.4  WIND-SHEAR  GUIDANCE  SYSTEM  CERTI¬ 
FICATION 

In  the  past  several  years,  simulators  have  been  used  in 
the  certification  of  warning  and  guidance  systems 
designed  to  assist  the  pilot  in  encounters  w  ith  severe 
wind-shear  conditions.  Since  their  critical  function  is  in 
an  extremely  rare  (but  extremely  dangerous)  environ¬ 
ment,  simulation  offers  the  only  practical  means  to 
demonstrate  performance.  In  this  case,  the  quality  of  the 
aircraft  modeling  is  not  of  as  much  concern  as  the  qual¬ 
ity  of  sensor  and  environment  modeling,  although  there 
must  be  some  confidence  in  the  modeling  of  aircraft 
response  to  the  severe  environments.  Of  course,  the 
flight  hardware  is  used  in  the  final  simulator  evalu¬ 
ations.  Again,  the  certifying  authorities  are  requiring 
documentation  of  initial  operational  experience  with  the 
systems  in  attempts  to  verify  the  simulation  experiences. 

A.5  HEAD-UP  DISPLAY  CERTIFICATION 

In  1986.  a  head-up  flight  guidance  display  system 
(HUD),  utilizing  conformal  "flight  path"  symbology, 
was  certificated  for  use  in  manual  "CAT  Ilia"  landings 
in  the  727-100  aircraft.  This  certification  permits 
manual  operation  in  visibility  that  require  automatic 
landing  with  conventional  cockpit  guidance  displays. 
Recognizing  that  piloted  tests  to  obtain  landing 
performance  data  of  a  statistical  nature,  of  the  scope 
obtained  in  non-real-time  simulation  of  automatic-land¬ 
ing  systems,  would  be  impractical,  the  certifying 
authorities  accepted  more  limited  data  from  piloted 
simulation  in  the  determination  of  basic  landing 
performance  envelopes.  These  data  were  then  verified 
in  relatively  limited  flight  tests,  including  some  under 
the  actual  low-visibility  conditions.  The  developmental 
testing  of  the  system  had  been  conducted  in  a  Boeing 
Company  engineering  development  simulator 
considered  to  be  well  verified  in  a  727  configuration. 
The  certification  exercises  were  conducted  in  this  same 
simulator  using  the  flight  HUD  hardware  and  software. 
Important  in  these  simulator  tests  w'as  the  availability'  of 
a  day  or  night  visual  simulation  system  with  which 
realistic  low-visibility  conditions  could  be  presented. 
More  recently,  the  process  has  been  repeated  for  the 
727-200  aircraft  in  preparation  for  utilization  of  the 
system  in  twenty  of  the  aircraft  operated  by  a  US 
airline.  In  this  case,  initial  manual  operation  in  CAT  II 
is  approved,  with  (he  expectation  of  further  certification 
to  the  low'er  visibility  of  CAT  Ilia,  or  beyond,  as  opera¬ 
tional  experience  grow's. 
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Appendix  B 


EXAMPLES  OF  THE  USE  OF  SIMULATION  IN 
THE 

CERTIFICATION/ACCEPTANCE  OF  MILITARY 
AIRCRAFT  SYSTEMS 


B.l  ACCEPTANCE  TESTING  OF  SPACE 

SHUTTLE  LANDING  CONTROL  SYSTEMS 

Long  before  the  first  orbital  flight,  a  number  of  non- 
real-time  computer  simulations  and  piloted  simulations 
were  contributing  to  the  vehicle's  development.  The 
control  systems  and  navigation  systems  sub-contractors 
could  do  their  basic  development  work  with  non-piloted 
simulations,  but  Rockwell,  and  NASA  at  the  project 
headquarters  at  Johnson  Space  Center,  employed  a  mix 
of  piloted  simulators  for  integrated  systems 
development  and  testing.  Some  of  these  used  elements 
of  flight  hardware.  In  the  process  of  verifying  the 
selection  of  the  basic  control  modes  for  approach  and 
landing,  in  1975,  the  piloted  simulation  research 
facilities  at  NASA's  Ames  Research  Center  were 
employed.  Since  that  time.  Ames  facilities  have  been 
used  in  support  of  Shuttle  landing  systems'  development 
and  acceptance  on  a  regular  basis.  On  several  occasions, 
an  in-flight  simulator,  the  Calspan  TIFS  aircraft,  was 
flown  in  simulation  of  Shuttle  landings.  Experiences 
W'ith  several  of  these  simulations  are  recounted  in  the 
following  paragraphs  as  examples  of  relatively 
successful  use  of  the  medium. 

B.1.1.  Configuration  Control. 

It  was  obvious  that  simulation  model  configuration 
control  and  verification  across  this  mix  of  facilities  w-ere 
required.  Basic  sources  for  the  aerodynamic  model,  the 
control  and  navigations  systems  models,  and  the  atmos¬ 
pheric  and  environmental  models  were  identified.  These 
sources  implemented  the  necessary  updates  to  the 
models  as  flight  data  was  obtained,  and  provided  the 
library  of  static  and  dynamic  check  cases  to  be  used  in 
the  re-verification  of  all  of  the  simulations. 

B.1.2.  Basic  Control  System  Assessments. 
Rate-command  roll  and  pitch  systems  were  chosen  and 
tuned  on  the  basis  of  fixed-cockpit  simulation.  Further 
acceptance  tests  were  conducted  in  an  Ames  simulation, 
which  at  that  time  incorporated  large-amplitude  lateral 
motion  but  very  modest  vertical  motion.  Cockpit  motion 
exposed  an  unacceptable  lateral  lurching  that  accompa¬ 
nied  roll  control  inputs,  the  result  of  an  overly  "tight" 
roll-rate  command  system  and  a  pilot  location  well 
above  the  roll  axis.  A  significant  increase  in  the  roll- 
response  time  constant  alleviated  the  problem  without 
otherwise  degrading  lateral  handling  qualities.  The 
pitch-rate  command  mode  w  as  unfamiliar  for  flare  and 
landing,  and  the  geometry  and  mass  distribution  of  the 


vehicle  combined  to  produce  an  unusual  lagged  heave 
response  to  pitch.  With  familiarization,  the  pilots  were 
able  to  demonstrate  precise  touch-downs  with  a  pitch- 
control  technique  characterized  by  intermittent  pulsing 
of  the  hand  controller.  In  the  formal  evaluations,  the 
pitch  control  system  was  assessed  as  satisfactory  for  the 
glide  flight  tests;  however,  in  less  formal  flying  of  the 
simulation  in  off-nominal  flare  trajectories,  several  brief 
encounters  with  pitch  PIO  (pilot-induced  oscillation) 
w  ere  noted.  These  events  w'ere  not  seriously  considered 
by  the  primary  evaluators,  who  tended  to  attribute  them 
to  the  inadequacies  of  simulation  and  inappropriate 
piloting  technique.  On  the  fifth  test  flight  from  the  747, 
as  the  pilot  was  attempting  a  precision  touch-down  on 
the  runway  instead  of  the  relatively  limitless  dry-lake 
bed.  he  found  himself  in  a  serious  PIO  that  terminated 
only  when  he  removed  his  hand  briefly  from  the 
controller.  Bom  was  a  new  objective;  develop  and  test 
control  system  modifications  aimed  at  eliminating  PIO 
tendencies. 

B.1.3  PIO  Suppression. 

For  this  effort,  two  unique  simulation  facilities  were 
employed.  At  Ames,  the  new  Vertical  Motion  Simulator 
(VMS)  offered  large  amplitude  vertical  cockpit  motion 
(16  meters),  increasing  the  fidelity  of  the  simulation  of 
longitudinal  maneuvers;  and  the  TIFS  aircraft  was 
brought  into  the  study.  These  facilities  were  used  to 
evaluate  the  effectiveness  of  practical,  rather  modest 
control  system  software  modifications.  One  concept  that 
was  retained  was  a  "PlO-suppressor"  logic  which,  upon 
identifying  the  onset  of  large  cyclic  controller  inputs, 
reduced  the  effective  output  gain  from  the  controller  to 
prevent  further  amplification  of  the  vehicle  motion. 

Simulation  studies  of  the  Shuttle  longitudinal  control 
system  were  given  high  priority  during  this  period  and 
through  the  years  of  the  early  orbital  test  flights,  and 
confidence  in  the  control  systems,  and  the  simulation 
assessment  procedures,  increased.  However,  some  basic 
truths  about  the  use  of  piloted  simulation  were  demon¬ 
strated.  The  difficulty  of  simulating  PIO  control 
problems  is  discussed  in  Section  3. 1. 1.2  of  this  paper. 

B.1.4.  Auto-land  System  Verification  and 
Acceptance. 

From  the  initial  concept,  the  Shuttle  navigation  and 
control  systems  were  to  include  an  automatic-landing 
mode;  however,  its  eventual  utilization  as  the  primary 
mode  of  flight  was  a  matter  of  controversy  between 
project  management  and  crew.  As  with  all  automatic 
systems,  the  primary'  simulation  efforts  were  non-real¬ 
time  computer  studies  of  performance  in  the  imaginable 
ranges  of  environments  and  component  performance. 
The  role  of  piloted  simulation  in  acceptance  w  as  limited 
to  two  issues;  definition  of  pilot-acceptable  levels  of 
degraded  performance  of  the  critical  sensors,  and  pilot 
capabilities  to  gracefully  revert  to  manual  control  in  the 
event  of  automatic-landing  system  failures  at  critical 
points  in  the  landing  trajectory.  These  verification 
efforts  in  ground-based  simulation  were  complemented 
by  experiences  with  brief  engagements  of  the  system 
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during  recoveries  from  the  early  orbital  flights,  and 
some  work  with  the  Shuttle  Training  Aircraft  (STA). 
But.  of  course,  it  was  only  in  the  ground-based 
simulator  that  tire  necessary  breadth  of  combinations  of 
system  failures  and  flight  conditions  could  be  presented 
to  the  pilot.  Pilot  acceptance  of  the  simulation  of  the 
landing  task  as  valid  for  this  type  of  assessment  was  a 
basic  requirement.  Fortunately,  experience  in  the  early 
orbital  flights  helped  to  confirm  that  the  motion  and 
visual  cueing  capabilities  in  the  VMS  facility  were 
providing  an  effective  representation  of  the  Shuttle 
landing  task.  The  pilots  concluded  their  assessments 
with  the  recommendation  that  the  automatic-landing 
system  should  not  be  certified,  not  because  of  marginal 
performance,  but  because  they  felt  that  the  final  flare 
trajectory'  and  the  characteristics  of  the  manual  pitch 
control  system  combined  to  present  an  unacceptable 
level  of  risk  with  even  the  modest  transients 
experienced  at  reversion  to  manual  control.  Their 
conclusions  also  reflected  their  concerns  regarding 
possible  degraded  pilot  performance  following  the 
rigors  of  orbital  flight  and  high-g  re-entry. 

B.1.5.  Steering  and  Braking. 

During  the  past  four  years,  most  of  die  piloted  Shuttle 
simulation  conducted  at  Ames  have  been  devoted  to  a 
study  of  the  problems  of  control  on  the  runway  from 
touchdown  to  final  stop.  If  the  Shuttle  is  to  use  the 
runway  at  Kennedy  Space  Center,  it  must  be  able  to 
deal  with  the  possibility  of  significant  crosswind,  and  to 
cope  with  tire,  brake  or  nose-wheel  steering  failures. 
The  economics  of  space  flight  dictate  that  the  landing 
gear  svstems  be  kept  to  a  minimum  weight,  and  thus  to 
minimum  performance  margins.  Concerns  regarding  the 
landing-gear  systems  were  amplified  by  incidents  seen 
in  the  first  runway  landings,  and  it  was  decided  that  the 
dry  lake  bed  at  Edwards  AFB.  where  cross-wind  land¬ 
ings  could  be  avoided,  would  remain  the  site  of  Shuttle 
landings  until  the  steering  and  braking  systems  could  be 
improved  and  certified. 

The  validity  of  the  VMS  simulation  of  the  piloted  land¬ 
ing  through  normal  touchdown  and  de-rotation  was 
accepted,  but  it  was  recognized  that  the  representation 
of  steering  and  braking  tasks,  particularly  with 
simulated  nose-wheel  steering  or  tire  failures,  was 
completely  dependent  upon  the  accuracy'  of  the 
modelling  of  the  tire  side-force  characteristics.  During 
the  course  of  the  simulations,  doubt  regarding  the  tire 
modelling  prompted  extensive  measurements  of  Shuttle 
tire  characteristics  on  the  NASA-Langley  test  track.  The 
new  data  significantly  changed  the  behavior  of  the 
simulated  vehicle  in  critical  scenarios.  The  simulations 
provided  a  wealth  of  data  regarding  tire  and  braking 
loads  as  they  are  affected  by  the  wind  environment, 
failure  modes  and  pilot  technique,  and  lead  to  the 
conclusion  that  the  modified  systems  are  adequate  for 
cross-wind  landings  on  the  runway.  Final  certification  of 
die  system  modifications  is  being  sought  by  analysis  of 
data  from  the  first  flights  in  the  resumed  series.  If  the 


results  are  as  expected,  landings  at  Kennedy  Space 
Center  w'ill  be  resumed. 

B.2  CLEARANCE  TESTING  OF  PAN  AVIA 
TORNADO 

A  brief  surv  ey  is  given  below  of  the  main  simulation 
activities  concerning  certification,  acceptance  testing 
and  validation  during  the  development  of  Tornado. 
Simulation  means  in  this  case  "manned  simulation"  w  ith 
pilots  or  simulation  engineers  testing  in  real-time.  The 
main  items  are: 

•  simulation  for  flight  control  system  develop¬ 
ment. 

•  simulation  for  flight  mechanics  work,  and 

•  autopilot  and  terrain-following  simulation. 

During  the  development  phase  of  the  Tornado  weapon 
system,  extended  simulations  were  carried  out  at  MBB 
for  the  definition  and  optimization  of  subsystems,  inte¬ 
gration  and  ground  testing  of  systems  and  flight  test 
support.  Three  main  factors  determined  the  simulation 
work: 

•  MBB’s  main  responsibility  for  the  control  and 
stability  augmentation  system  (CSAS) 

•  Main  responsibility  for  the  basic  autopilot 
modes  and  at  a  later  stage  for  the  complete 
autopilot 

•  Main  responsibility  for  terrain-following  flight 
testing. 

Validation  of  the  simulation  w  as  performed  mainly  by 
comparing  simulation  results  with  partner  companies 
and  suppliers.  A  special  validation  procedure  was 
conducted  for  the  safety-critical  flight  phase  of  terrain 
following. 

At  the  start  of  Tornado  development,  all  systems  were 
modelled  in  the  simulation  computer.  A  flight  control 
rig  and  an  avionic  rig  containing  real  aircraft  hardware 
were  coupled  later  with  the  simulation.  Investigations 
where  hardware  rigs  are  coupled  with  simulators 
increase  the  complexity  but  also  increase  confidence  in 
the  results. 

B.2.1  Flight  control  System. 

Investigation  into  failure  behaviour  and  switch-over 
transients  is  one  of  the  most  important  fields  of  simula¬ 
tion.  When  flight  testing  has  started,  the  risk  of  testing 
failure  cases  in  flight  strongly  supports  the  extensive  use 
of  ground-based  simulation,  especially  in  coupled 
simulation  with  the  flight  control  rig.  Some  examples  of 
failure  simulations  are: 

•  CSAS  failures  and  switch-over  to  reduced 
modes. 

•  Engine  failure  with  maximum  oil  flow  reduc¬ 
tion. 

•  Failure  of  a  control  surface  or  actuator. 
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Acceptance  testing  of  the  complete  flight  control  system 
hardware  included  a  "confidence  test"  to  prove  the 
lifetime  of  the  system  components  and  to  improve  the 
confidence  level  when  preparing  the  flight  tests.  A 
typical  one  hour  flight  containing  various  manoeuvres 
was  simulated  on  the  flight  control  rig  and  recorded  on 
tape.  Then  the  systems  on  the  rig  were  stimulated 
several  hundred  times  with  the  recorded  signals  and 
parameters.  The  total  testing  time  on  the  rig  had  to  be 
considerably  longer  than  the  total  flight  time  of  the  test 
aircraft. 

B.2.2  Flight  Mechanics. 

Handling  quality  assessment  was  supported  by  simula¬ 
tion  flights  over  the  entire  flight  envelope,  which 
rapidly  indicated  problems  and  shortcomings  which  had 
to  be  improved.  This  procedure  was  repeated  for  all 
critical  store  configurations.  Where  the  specification 
required  "acceptable"  handling  qualities  without 
mathematical  definition,  simulation  was  the  only  means 
of  proving  the  requirement  prior  to  flight  testing.  Glide 
path  stability  during  approach,  touch-down,  landing  and 
ground  roll  were  investigated  with  and  without  cross- 
wind  and  gusts.  Ground  roll  with  thrust  reverse  required 
the  development  of  a  nose  wheel  steering  augmentation 
system  using  simulation  facilities  extensively. 

B.2.3  Autopilot/Terrain-Following  System. 

The  simulation  work  on  the  Autopilot  and  on  the 
Terrain-Following  System  mainly  concerned  failure 
investigations  and  harmonisation  with  the  other  parts  of 
the  system,  such  as  the  CSAS  and  the  electric  and 
hydraulic  systems.  A  survey  of  this  work  is  given  in 
reference  30. 

The  certification  of  the  terrain  following  system  w7as 
performed  in  close  cooperation  between  flight  test  and 
simulation  departments.  It  is  difficult  and  time-consum¬ 
ing  to  define  the  proper  flight  path  of  a  TF  flight.  There¬ 
fore  a  detailed  comparison  between  one  flight  and  a 
corresponding  simulation  flight  was  performed.  The  two 
flight  paths  were  matched  until  relatively  small  and 
explicable  differences  wrere  reached31. 

Simulation  flights  over  various  flight  test  terrains 
proved  that  the  required  criteria  in  tire  specification 
were  met,  such  as  peak  performance,  clearance  height 
error,  maximum  undershoot,  safety  factor  and  so  on.  For 
TF  performance  evaluation,  simulation  is  considered  as 
the  adequate  reference.  When  preparing  flight  testing, 
flight  clearances  had  to  be  reached  in  extended  tests  and 
also  in  simulation  tests,  mainly  in  conjunction  with  the 
flight  control  rig  and  avionic  rig.  Autopilot  and  terrain- 
following  flight  testing  is  safety-critical,  and  a  special 
company  based  qualification  procedure  gave  confidence 
as  regards  the  correctness  of  the  rig  simulation32. 

B.3  USE  OF  SIMULATION  FOR  ACCEPTANCE 
TESTING  OF  DASSAULT/DORNIER  ALPHA  JET 

A  brief  survey  is  given  below7  of  the  main  activities 
concerning  acceptance  testing  and  validation  during  the 
development  of  Alpha  Jet. 


The  Alpha  Jet  was  specified  as  a  training  aircraft  and 
had  therefore  -  w'ith  the  exception  of  a  yaw  damper  -  a 
very  conventional  control  system.  It  was  developed  in 
collaboration  between  Aviation  Marcel 
Dassault/Breguet  Aviation,  France  and  Domier. 
Germany.  Here  only  the  simulation  work  at  Domier  is 
described. 

For  certification/acceptance  testing,  simulation  at 
Domier  was  used  for  special  topics  in  flight  mechanics 
and  as  support  for  the  flight  tests.  The  simulation 
facility  used  was  a  fixed  based  operation  flight  and 
tactics  simulator  which  is  now  used  for  development 
and  training  of  air-to-surface  and  air-to-air  missions. 
During  the  development  of  the  aircraft  the  simulation 
equipment  was  often  used  without  a  pilot  in  the  loop, 
but  with  stylised  control  inputs  and/or  w'ith  a  'paper1 
pilot. 

The  main  simulator  tests  concerned  roll  coupling 
effects.  In  many  simulation  runs  through  the  whole 
flight  envelope  it  was  demonstrated  that  there  were  no 
problems  at  all  caused  by  roll  coupling  effects.  This  had 
to  be  proved  before  flight  tests  with  the  real  aircraft 
started. 

Another  group  of  certification  tests  using  simulation 
concerned  the  usual  problems  in  flight  dynamics  which 
could  not  be  handled  with  linearised  equations,  that  is 
problems  where  the  theory  of  small  disturbances  cannot 
be  applied.  For  the  Alpha  Jet,  there  were  simulations  to 
examine 

•  engine  failure 

•  longitudinal  dynamic  behaviour. 

In  these,  the  simulated  aircraft  was  controlled  by  a 
'paper'  pilot.  The  engine  failure  investigations  had  to 
prove  that  transient  behaviour  could  be  handled  by  a 
trainee  pilot. 

The  simulations  concerning  longitudinal  dynamic 
behaviour  had  to  prove  that  there  are  no  difficulties  for 
a  trainee  pilot  with  changes  in  pitching  moment  due  to 
landing  flaps  and/or  changes  in  engine  power. 

Furthermore,  simulation  w7as  used  to  support  the  Alpha 
Jet  flight  testing.  An  important  aspect  of  this  work  was 
to  ensure  that  the  simulator  and  the  aircraft  had  the 
same  behaviour.  A  "certification"  of  the  mathematical 
model  was  obtained  wdiich  could  then  be  used  in  the 
training  simulators. 

B.4  AERITALIA/AEROMACCHI  AMX  VALIDA¬ 
TION 

B.4.1  Clearance  at  High  Incidence 

After  an  initial  effort  to  evaluate  the  vertical  wind 
tunnel  results  through  an  assessment  in  the  flight 
simulator,  a  few  flights  were  conducted  to  ascertain  the 
aircraft  responses  from  high  alpha,  through  a  transition 
phase,  into  the  departure  and  spin. 
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While  vertical  tunnel  data  are  of  good  value  to  assess 
aircraft  responses  in  a  developed  spin,  after  the  depar¬ 
ture.  they  are  of  little  value  to  investigate  the  initial 
departure  phase.  Therefore  the  first  batch  of  flights  were 
done  principally  to  investigate  this  phase  more  thor¬ 
oughly.  Once  enough  confidence  was  achieved  and 
enough  time  had  been  spent  in  the  simulator  by  the 
project  pilots  to  gain  familiarity'  with  aircraft  responses 
and  to  ascertain  that  the  simulator  standard  was  close 
enough  to  the  aircraft  behaviour,  a  further  step  was 
taken.  Manoeuvres  involving  high  alpha,  high  beta, 
departures  and  recoveries  were  assessed  thoroughly  in 
the  simulator  and  repeated,  step  by  step,  in  the  air. 

This  led  to  a  situation  in  which  actual  flight  data  were 
continuously  fed  into  the  simulator  model  and  allowed 
an  even  deeper  understanding  of  the  flight  mechanics 
involved  in  the  AMX  spin  characteristics  to  be  obtained. 

Tire  evaluation  of  motion  cross-coupling,  and  the  alpha, 
beta  and  roll  dynamic  interchanges,  were  assessed  for 
tens  of  seconds,  thus  allowing  fully  developed  spins  to 
be  flown  safely.  When  the  benign  situation  of  the  basic 
configuration  w'as  completely  known,  a  more 
demanding  phase,  from  the  point  of  view  of  simulation 
and  prediction,  was  initiated.  This  phase  is  related  to  the 
external  stores  configurations. 

The  first  effort  was  to  determine  significant  external 
store  configurations,  so  that  test  results  from  a  minimum 
number  of  configuration  would  be  enough  to  assess  the 
totality  of  combinations.  In  this  context,  a  real  problem 
was  found,  during  preliminary  simulation  assessment, 
for  some  asvmmetric  configurations:  if  pro  spin  controls 
were  held  for  a  sufficient  time  (10-20  sec)  the  dynamics 
would  bring  the  aircraft  into  a  flat  spin  with  extremely 
high  yaw  rates:  a  practically  non-recoverable  situation. 
When  this  was  discovered,  special  flight  tests  w'ere  used 
to  collect  specific  data  to  further  refine  the  model. 

Simulation  still  shows  a  potentially  dangerous  situation 
if  the  aircraft  is  allowed  to  get  into  a  flat  spin.  But 
enough  was  learnt  in  the  simulation  to  allow  a  deeper 
in-flight  evaluation  and.  possibly,  to  pursue 
modifications  of  the  FCS  and/or  store  configurations  to 
obtain  a  departure  proof  aircraft,  or  at  least  spin 
resistance  in  ALL  configurations.  In  any  case,  the 
results  of  this  testing  have  been  so  encouraging  that  it 
has  been  decided  that  simulation  will  be  used  for 
clearing  the  two  seater  version  of  AMX  to  perform 
deliberate  spinning. 

The  process  of  validation,  as  it  stands  today,  foresees  a 
limited  number  of  flights,  and  attention  to  obtain  good 
matching  with  simulation.  From  this  point,  all  assess¬ 
ment  and  validation  will  be  done  in  the  simulator,  with 
only  a  few'  selected  departures/spin  manoeuvres 
reassessed  in  flight. 

This  use  of  the  simulator  can  be  considered  a  proof  of 
concept  attempt  to  obtain  certification  through  simula¬ 
tion:  if  it  can  be  done  for  high  alphas,  and  departure  and 


spinning  of  an  aircraft,  it  can  be  applied  to  any  other 
field. 

B.4.2  Air  Brake  Development 

This  example  is  related  to  the  unusual  air  brake  configu¬ 
ration  (for  a  fighter  aircraft),  in  which  the  surfaces  are 
used  also  for  lift  dumping.  Their  actuation  caused  a 
pronounced  nose  down  trim  change,  and  this  was 
unacceptable  from  an  operational  point  of  view 
(formation  flight,  low'  level  flying,  etc). 

It  w'as  therefore  decided  to  compensate  this  bchaviour 
by  automatic  scheduling  of  the  stabilator.  The 
scheduling  was  designed  to  maintain  a  straight 
trajectory  after  airbrake  actuation  and  this  required  the 
incidence  to  be  increased  by  between  1  and  3  degrees. 
Simulation  showed  that  this  attitude  change  could  be 
unacceptable  if  done  in  an  open  loop  situation.  Also,  the 
immediate  decreasing  of  airspeed  caused  a  natural  nose 
dowm  tendency,  and  this  seemed  to  be  objectionable. 
Before  going  on  and  introducing  the  software  change 
(and  the  ensuing  validation  process,  lasting  7-10 
months),  a  series  of  simulation  tasks  were  defined, 
especially  to  ascertain  the  acceptability  of  the  closed 
loop  case. 

Close  formation  flying  on  a  very  detailed  aircraft  model 
was  done  on  the  simulator,  because  this  was  thought  to 
be  the  most  demanding  operational  task  as  far  as  trim 
change  was  concerned.  Formation  flight  was  not  only 
simulated  on  aircraft  of  the  same  type  as  the  one  tested, 
but  also  on  different  tactical  aircraft  models.  It  was 
found  that  the  closed  loop  case  was  fully  acceptable 
throughout  the  operational  flight  envelope. 

The  customers’  assessment  of  the  modification  (OTC 
pilots)  w'as  not  so  favourable.  However,  the  findings  of 
the  in-house  evaluation  were  thought  to  be  good  enough 
to  proceed  with  the  changes  in  FCS  software.  Subse¬ 
quent  flight  test  confirmed  the  effectiveness  of  the 
modification  and  also  gained  approval  from  customers. 
It  was  found  that  insufficient  familiarity  with  the  real 
aircraft  response,  and  insufficient  knowledge  of  the 
simulator,  caused  the  different  judgements  of  these 
changes. 

B.4.3  Gun  Effectiveness 

This  example  shows  a  situation  in  which  the  FCS  soft¬ 
ware  changes  developed  in  the  simulator  were  accepted 
both  during  the  in-house  and  customer  evaluations. 
Flight  tests  on  the  aircraft  showed  that  the  asymmetric 
position  of  the  gun  in  the  nose  caused  an  unwanted 
horizontal  dispersal  of  burst  during  strafing.  Traces  of 
actual  flight  tests  were  assessed  and  a  modification  to 
the  FCS  corrected  this  problem  via  automatic  rudder 
inputs.  Before  any  flight  verification,  this  modification 
has  been  accepted  bv  company  and  OTC  crews  through 
simulator  evaluation. 

The  behaviour  of  the  aiming  dot,  which  accurately 
represents  the  bullets'  impact  point  through  a  very 
sophisticated  fire  control  system,  showed  exactly  the 
problem  encountered  during  actual  flight  test  and  its 
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solution  through  FCS  software  modification.  This 
proved  sufficient  not  only  to  validate  the  effectiveness 
of  modification,  but  also  to  gain  approval  of  OTC  crew, 
and  therefore  official  customer  acceptance  of  the  gun 
aiming  system,  even  before  flight  tests  were  completed. 
The  reasons  of  this  very  favourable,  but  different,  result 
were  assessed:  it  was  discovered  that  the  flight  experi¬ 
ence  accumulated  during  the  gap  between  the  two 
studies  had  been  sufficient  for  the  OTC  crew  to  fully 
realise  the  validity  of  the  simulation. 

Achievements  of  this  type  are  dependent  on  the 
accuracy  of  the  process  of  validation  of  the  piloted 
simulator  system.  A  thoroughly  validated  simulation 
system  gives  the  possibility  to  discover  unexpected 
responses  throughout  the  full  flight  envelope. 

B.5  DESIGN,  DEVELOPMENT,  AND  CLEAR¬ 
ANCE  OF  BAe  EAP 

The  British  Aerospace  Experimental  Aircraft  Pro¬ 
gramme  (EAP)  is  a  technology  demonstrator  aircraft 
which  features  an  unstable  aircraft  configuration  aug¬ 
mented  by  Active  Control  Technology  to  achieve  satis¬ 
factory  flying  characteristics.  The  development  simula¬ 
tor  facilities  at  British  Aerospace's  Warton  plant  have 
been  used  throughout  the  development  and  clearance 
process  of  the  digital  fly-by-wire  flight  control  system. 
Initial  fundamental  design  work  was  carried  out  off-line. 
Very  early  in  tire  programme,  however,  it  was  essential 
to  carry  out  pilot  assessment  of  the  proposed  aircraft 
handling  characteristics. 

B.S.1  Origin  of  aerodynamic  data. 

Since  this  was  a  new  aircraft  configuration, 
aerodynamic  data  originated  from  wind  tunnel  testing, 
as  well  as  analytical  work.  These  were  combined  to 
produce  a  definitive  dataset  which  covered  the  flight 
envelope  of  the  aircraft.  Non-real-time  simulations  were 
constructed  on  a  mainframe  computer,  to  allow 
frequency  response  and  time  history  plots  to  be 
produced  off-line  to  assist  the  design  work. 

B.5.2  Origin  of  Flight  Control  System  Data. 

Design  studies  using  the  non-real-time  model  produced 
the  FCS  definition  which,  for  integrity  and  convenience 
reasons,  takes  the  form  of  Fortran  code  although  this  is 
not  the  language  used  in  the  airborne  computers.  Inter¬ 
faces  to  this  code  and  operating  constraints  were  also 
defined. 

B.5.3  Implementation  for  Real-time  Simulation. 

The  off-line  facilities  mentioned  above  are  subject  to 
their  own  independent  quality  audit  procedure.  The 
man-in-the-loop  simulator  basically  requires  a  real-time 
version  of  these  facilities.  To  achieve  this,  some  rework¬ 
ing  of  the  aerodynamic  data  is  required.  This  is  a 
rearrangement  of  data  into  more  convenient  groups, 
based  on  fixed  flight  conditions,  which  can  then  be 
interpolated  in  real-time.  Further,  data  is  often 
rearranged  into  regular  grids,  the  general  objective 
being  to  pre-process  data  as  far  as  possible  thereby 
alleviating  the  run-time  computing  task.  The  software 


which  uses  this  data  is  different  from  that  used  for  off¬ 
line  design  studies  and  has  evolved  over  a  period  a  time 
with  real-time  performance  being  a  prime  consideration. 
The  resulting  simulator  frame  rate  is  approximately 
300Hz.  The  FCS  Fortran  definition  code  is  simply 
compiled  and  executed  with  the  appropriate  interfaces 
to  the  aerodynamic  model,  and  at  the  defined  iteration 
rates. 

B.5.4  Validation. 

Cross  checking  w  ith  the  off-line  equivalent  model  is  the 
primary  method  of  validating  the  simulator.  Both  the 
off-line  and  real-time  facilities  use  different  software 
and  different  versions  of  the  same  aerodynamic  data  set 
and.  when  a  cross-check  match  is  achieved,  it  also 
provides  an  additional  level  of  confidence  in  the  validity 
of  both  models. 

The  steps  in  the  validation  process  are: 

a)  Static  checks  of  aerodynamic  model.  The 
model  is  forced  to  a  particular  flight  condition 
(airspeed,  altitude,  incidence,  sideslip,  g,  mass, 
centre  of  gravity  etc.)  and  the  aerodynamic 
force  and  moment  coefficients  are  extracted. 
The  software  used  for  this  is  as  close  to  the  run¬ 
time  standard  as  possible. 

b)  Static  checks  of  the  FCS  model.  Given  a  fixed 
set  of  sensor  and  control  demand  conditions, 
the  values  at  various  points  within  the  FCS 
software  network  are  extracted  and  cross 
checked. 

c)  Dynamic  checks  of  aerodynamic  model.  The 
unaugmented  aerodynamic  model  is  forced  by 
known  inputs  to  the  control  surfaces  and  a  time 
history  produced.  In  the  case  of  a  relaxed 
stability  aircraft  such  as  EAP  this  has  limited 
application  in  the  pitch  axis  because  of  its  rapid 
divergence  but  is  a  useful  cross  check  for  the 
remaining  stable  axes.  Again,  the  checks  are 
made  at  different  flight  conditions. 

d)  The  final  end-to-end  check  is  made  by  forcing 
the  complete  FCS  and  aerodynamics  model 
with  known  pilot  control  displacements.  As 
before,  time  histories  are  produced  for  various 
flight  conditions.  It  is  important  to  consider 
combined  controls  as  well  as  single  axis  inputs. 

B.5.5  FCS  Developments. 

The  simulator  model  validated  as  above  is  used  to  allow 
pilot-in-the  loop  assessments  as  part  of  the  design 
process.  Deficiencies  or  recommendations  are  reported 
back  to  the  designers  who  may  offer  a  revised  FCS.  This 
is  implemented  using  the  same  processes,  and  re¬ 
assessments  then  take  place.  Design  iterations  are 
carried  out  in  this  way  until  a  satisfactory'  standard  of 
FCS  is  achieved.  Any  refinements  in  aerodynamic  data 
are  introduced  as  necessary,  again  using  the  same 
validation  methods.  At  this  stage,  FCS  design  can  be 
considered  to  be  complete. 
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B.5.6  FCS  Clearance. 

Detailed  assessments  are  carried  out  through  a  fine 
matrix  of  flight  conditions,  up  to  and  beyond  the  limits 
of  the  flight  envelope,  using  both  the  design  dataset  and 
data  subjected  to  tolerances.  These  are  confidence 
checks  to  ensure  that  the  FCS  design  is  robust  within  its 
flight  envelope.  Beyond  the  envelope,  the  handling 
should  degrade  gracefully  without  'knife-edge'  situations 
developing.  Applying  tolerances  to  the  data  is  an 
additional  confidence  check,  which  ensures  that  the 
sensitivity  of  the  FCS  performance  to  design  data 
accuracy  is  what  was  anticipated. 

B.5.7  Pilot  Familiarisation. 

Lastly,  the  simulator  is  used  for  pilot  familiarisation. 
This  is  not  training  as  such  but  reminding  the  aircrew  of 
the  expected  aircraft  handling  prior  to  flight.  In  the  case 
of  a  completely  new  configuration  such  as  EAP,  the 
complete  first  flight  schedule  was  rehearsed  on  the 
simulator. 

As  the  above  discussion  indicates,  the  man-in-the-loop 
simulator  plays  a  vital  role  in  the  various  stages  of  the 
design,  development  and  clearance  of  an  FCS  for  an 
aircraft  such  as  EAP.  It  is  however  a  facility  which  is 
one  part  of  a  larger  set  which  includes  off-line  models 
as  well  as  test  rigs.  The  underlying  theme  throughout 
the  process  is  the  build  up  of  confidence  in  both  the 
aircraft  handling  qualities  and  the  facilities  used  to 
design  those  handling  qualities. 

B.6  GRUMMAN  F-14A  AUTOMATIC  RUDDER 
INTERCONNECT  (ART)  TESTING 

The  Naval  Air  Development  Center  has  developed, 
validated  and  refined  the  centrifuge-based  Dynamic 
Flight  Simulator  (DFS)  to  simulate  the  total  G-force 
environment  of  high-performance  military  aircraft.  The 
DFS  is  the  only  simulator  in  the  world  capable  of 
producing  the  pilot-controlled,  sustained,  high-G  envi¬ 
ronment  experienced  in  actual  air  combat  maneuvering 
engagements.  It  utilizes  a  50  foot  radius,  man-rated 
centrifuge  as  its  moving  base.  This  simulator,  with  its 
dual-gimballed  gondola,  can  provide  linear 
accelerations  in  any  of  three  mutually  perpendicular 
axes  (Gx.  Gy.  and  Gz)  up  to  a  magnitude  of  10  G  with 
onset  rates  as  high  as  3-5  G/sec.  The  DFS  has  been  used 
to  study: 

1)  the  debilitating  and  disorienting  environment 
imposed  on  a  pilot  during  out-of-control  and 
spinning  flight  regimes, 

2)  generic  high  angle  of  attack  stability  and 
control. 

3)  pilots'  physiological  response  to  high  G 
eyeballs  out  spin  maneuvers,  and 

4)  advanced  tactical  aircraft  control  configura¬ 
tions. 

Its  aerodynamic  and  control  system  models  and  cockpit 
configuration  can  be  readily  modified  to  study  handling 


qualities,  control  law  authority,  cockpit  displays,  and 
the  effects  of  acceleration  on  pilot  workload  and 
performance.  The  DFS  provides  a  means  to  safely 
address  design  concepts  on  the  ground  prior  to  flight 
testing. 

The  baseline  DFS  F-14A  application  program  included 
the  original  NASA-Langley  Automatic  Rudder  Inter¬ 
connect  (ARI)  control  system.  The  U.S.  Navy,  in  con¬ 
junction  with  Grumman  and  NASA  Dryden,  have  been 
developing  and  flight  testing  a  new'  Low  Speed  Cross 
Control  ARI  (ARI/LSXC).  This  new  ARI  is  similar  to, 
but  different  from  that  used  in  the  original  F-14  applica¬ 
tion.  The  system  includes: 

•  Slow'  speed  (Mach  less  than  0.4)  feedforward 
loops 

•  Gain  and  breakpoint  changes 

•  Cross  control  logic 

•  A  Deadband  in  the  ARI  yaw  rate  feedback 
terms 

•  Variable  gain  control  capability  (for  test 
purposes  only). 

The  original  ARI  code  was  updated  to  reflect  these  new 
capabilities.  The  Variable  Gain  Control  switch  settings 
and  an  ARI  ON/OFF  switch  were  implemented  to 
provide  the  experimenter  with  control  of  the  ARI. 

The  implemented  ARI  consists  of  5  basic  elements:  (1) 
automatic  differential  stabilizer  limiting;  (2)  automatic 
spin  prevention;  (3)  lateral  stick  to  rudder  interconnect, 
(4)  wing-rock  suppression,  and  (5)  low  speed/high  angle 
of  attack  (AOA)  cross  control  capability.  Elements  (1) 
and  (2)  are  designed  to  improve  the  departure/spin 
resistance  of  the  F-14.  Elements  (3)  and  (4)  are  primar¬ 
ily  high  AOA  flying  qualities  enhancement  features. 
Element  (5)  was  specifically  designed  to  provide  the 
pilot  with  increased  roll/yaw  maneuvering  capability  in 
the  low  speed/high  AOA  flight  regime. 

Validation  of  the  resulting  ARI  modifications  was 
conducted  via  both  batch  and  real-time  testing.  The 
batch  tests  consisted  of  functional  tests  to  investigate  the 
DFS  Automatic  Rudder  Interconnect  implementation 
versus  the  control  system  block  diagrams.  Following 
successful  completion  of  the  batch  tests,  real-time  tests 
were  conducted  to  investigate  the  simulated  aircraft's 
response  to  control  system  commands  at  high  angle  of 
attack. 

The  batch  tests  were  conducted  from  a  trim  condition  of 
0.5  Mach  at  30,000  feet  altitude,  with  all  Stability' 
Augmentation  Systems  (SAS)  and  ARI  ON.  Full  lateral 
stick  w'as  selected  to  command  a  roll,  followed  by 
smooth  aft  longitudinal  stick  application,  while  main¬ 
taining  the  lateral  stick  input,  to  increase  angle  of 
attack.  Commanded  control  surface  positions  were  then 
recorded  as  a  function  of  angle  of  attack  and  plotted 
against  the  control  system  design  parameters.  The  low 
speed  cross  control  logic  was  tested  from  a  similar  trim 
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condition.  In  this  case,  rudder  pedal  was  applied 
opposite  to  lateral  stick  in  an  effort  to  induce  a  more 
rapid  roll.  Longitudinal  stick  was  then  applied  to 
increase  the  angle  of  attack.  Again,  the  differential  tail 
and  rudder  deflections  followed  the  ARI  schedules  with 
angle  of  attack  until  the  cross  control  thresholds  were 
satisfied.  With  angle  of  attack  greater  than  30  degrees 
and  Mach  number  less  than  0.4.  the  cross  control  logic 
restored  full  commanded  surface  deflections.  Excellent 
agreement  between  the  recorded  data  and  the  control 
system  design  parameters  was  obtained,  thereby  validat¬ 
ing  the  functional  operation  of  the  simulated  ARI. 

The  Variable  Gain  Control  feature  was  also  validated 
during  both  batch  and  real-time  testing.  It  was  used  to 
isolate  individual  loops  of  the  control  system  during 
software  unit  testing  and  verification,  as  well  as  to 
modify  control  gains  in  response  to  on-going  flight  test 
development. 

Satisfactory  implementation  of  the  DFS  simulated  F-14 
ARI  provided  the  NAVAIRDEVCEN  with  additional 
capability  to  support  development  of  the  F-14's  fight 
control  system  in  a  safe,  efficient  manner  under  realistic 
sustained-g  maneuvering  conditions  prior  to  a  flight  test. 

B.7  HIGH  AGILITY  FIGHTER  AIRCRAFT  -  X-31 

The  advent  of  all-aspect  capability  air-to-air  weapons  is 
causing  changes  in  the  air  combat  maneuvering 
doctrines  of  tactical  fighter  aircraft.  These  new  weapons 
-  including  radar  controlled  guns  in  combination  with 
pointing  flight  modes  -  provide  improvements  in  air 
combat  capability  via  improved  energy 
maneuverability.  New  maneuver  modes  are  required  to 
provide  the  pilot  with  substantial  improvements  in 
close-in  combat  effectiveness.  One  such  mode  involves 
Post  Stall  Maneuvering  (PST). 

The  U.S.  Defense  Advanced  Research  Projects  Agency 
(DARPA)  in  conjunction  with  the  U.S.  Navy  and  the 
West  German  government  has  developed  a  prototype 
aircraft  designated  the  X-31  whose  mission  is  to  investi¬ 
gate  the  technology  of  post-stall  (PST)  maneuvering  and 
enhanced  fighter  maneuverability  (EFM).  PST/EFM 
entails  flight  at  high  angles  of  attack,  beyond  aerody¬ 
namic  stall,  in  which  the  aircraft  is  rotated  about  the 
wind  vector  through  the  use  of  thrust  vector  control. 
Such  a  maneuver  enables  the  pilot  to  gain  a  tactical 
advantage  over  an  adversary  by  pointing  his  aircraft  to 
acquire  a  firing  opportunity  more  rapidly  than  can  his 
opponent. 

The  NAVAIRDEVCEN  participated  in  an  EFM  flight 
simulation  program  using  its  Dynamic  Flight  Simulator 
(DFS).  The  program's  purpose  was  two-fold:  (1)  to 
assess  the  DFS's  ability  to  duplicate  the  unusual 
disorienting  aspects  of  the  PST  environment  and  (2)  to 
perform  basic  human  factors  experiments  concerning 
EFM  control/display  concepts. 

Specific  areas  of  concern  in  this  simulation  included:  (1) 
the  effect  of  sustained  G,  G-onset  rates,  angular  accel¬ 


erations  and  acceleration  rates  on  the  pilot/aircraft 
combination,  (2)  the  need  for  special  PST  displays,  and 
(3)  trade-offs  in  control  laws  and  control  system 
mechanization. 

The  flight  control  laws  in  the  DFS  EFM  simulation  went 
through  a  number  of  revisions  during  this  program. 
Modifications  were  added  to  the  physical  characteristics 
of  the  cockpit  controls,  the  control  functions  and  the 
flight  control  laws  themselves.  These  modifications 
were  accomplished  to  provide  the  pilots  with  the  ability 
to  evaluate  alternative  means  of  conducting  the  PST 
maneuver  (i.e.,  use  of  lateral  stick  and/or  rudder  pedals) 
to  provide  a  thrust  vectoring  capability  for  control  at 
high  angles  of  attack  to  reduce  their  workload  in  execut¬ 
ing  the  PST,  and  to  improve  their  performance  in  the 
PST. 

Execution  of  the  PST  maneuver  in  the  DFS  EFM 
aircraft  required  the  utilization  of  pitch  vanes  to 
establish  and  maintain  the  extremely  nose  high 
attitudes,  and  yaw  vanes  to  turn  the  simulated  aircraft  at 
high  angles  of  attack.  The  pitch  vanes  w'ere  controlled 
via  the  longitudinal  stick  as  a  function  of  stick  position 
aft  of  the  stick  detent  position.  The  yaw  vane  was 
implemented  via  either  the  rudder  pedals  or  the  lateral 
stick.  The  DFS  crewstation  was  modified  to  provide 
both  angle  of  sideslip  and  angle  of  attack  indicators  to 
improve  the  pilot's  ability  to  control  the  airplane  at  high 
angles  of  attack. 

Based  on  the  results  of  this  study,  the  DFS  wns  found  to 
be  a  unique  laboratory  for  the  study  of  high  agility 
aircraft.  The  simulated  post  stall  maneuvers  were 
consistent  and  repeatable  and  induced  no  gross  ill 
effects  based  on  linear  or  angular  accelerations.  New 
approaches  to  cockpit  controls  and  displays  for  this  class 
of  aircraft  were  successfully  evaluated  to  determine 
their  value  in  improving  the  pilot's  attitude  awareness 
and  performance. 
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Appendix  C 


SOFTWARE  CONFIGURATION  MANAGEMENT 


Configuration  control  procedures  should  be  defined  in  a 
Configuration  Management  Plan.  This  plan  should 
define  personnel  and  organisational  responsibilities, 
configuration  management  strategy  and  scope,  and 
operational  procedures  (including  support  tools).  The 
international  standard  ISO  9000  provides  further 
guidance. 

In  general,  the  operational  procedures  should  embody 
the  following  features.  However,  some  of  the  features 
identified  below  are  regarded  as  desirable  rather  than 
essential  (denoted  by  *)  and  may  not  be  readily 
supported  by  proprietary  software  management  tools  for 
specific  applications,  eg  due  to  programming  language 
or  system  characteristics. 

a)  File  management. 

1.  Each  file  should  have  a  unique  name  and  a 
designated  owner  (a  person). 

2.  Files  (source,  'include'  and  object)  should  be 
renamed  or  copied  to  a  protected  directory  (by 
an  authorised  user). 

3.  The  protected  directory  should  be  backed  up 
regularly,  and  the  identifiers  of  the  off-line 
storage  media  used  should  be  recorded. 

4.  Files  should  have  the  option  of  password 
protection. 

5.  Files  in  the  protected  director}  should  be  read¬ 
only  accessible  to  all  users  (using  password  if 
appropriate). 

*  6.  File  dependencies  should  be  formally  identified. 

b)  Configuration  definition 

1.  Each  configuration  definition  should  have  a 
designated  owner. 

2.  A  configuration  definition  should  itself  be 
subject  to  configuration  management  in  the 
same  way  that  files  are  managed. 

3.  It  should  be  possible  for  any  file,  at  a  specified 
version,  to  be  included  in  any  configuration. 

c)  Version  identification 

1.  All  files  and  configuration  definitions  should 
employ  a  clear,  structured  version  numbering 
scheme. 

2.  The  identification  scheme  should  identify  the 
file  or  configuration's  state,  ie  baseline  or  under 
development/not  tested. 


d)  Change  control 

1.  Each  change  specification  should  have  a  unique 
identifier  and  a  designated  owner  (the  person 
who  is  authorised  to  refine  the  change 
specification  and  responsible  for  the  work). 

2.  Each  change  specification  should  clearly  state 
the  reason  for  the  change  and  the  files  or 
configuration  definitions  affected. 

3.  Each  change  specification  should  be  stored  in  a 
protected  area. 

4.  All  users  affected  by  a  proposed  change  should 
indicate  their  agreement  with  the  change 
specification  before  it  is  actioned. 

5.  Once  the  change  specification  is  agreed,  it 
should  not  be  modified  further. 

6.  Parallel  changes  to  the  same  file  or 

configuration  definition  should  not  be  allowed. 

7.  An  authorised,  independent  user  should  approve 
the  implementation  of  the  change  before  the  file 
or  configuration  definition  is  re-baselined.  The 
approval  details  should  be  logged  in  a  protected 
area. 

8.  The  date  and  change  identifier  should  be 
recorded  for  each  version  of  a  file  or 
configuration  definition. 

*  9.  It  should  be  possible  to  log  software  problems 

for  which  change  specifications  will  be  raised 
in  the  future. 

e)  Development  support. 

*  1.  When  a  file  is  modified,  all  the  files  that 

depend  on  that  one  should  be  automatically 
updated. 

*  2.  Pre-processors,  compilers,  linkers,  utilities,  etc. 

should  be  automatically  invoked. 

*  3.  The  specified  file  dependencies  should  define 

the  order  in  which  the  files  are  automatically 
processed. 

*  4.  Default  processing  options  should  be 

automatically  specified  for  the  files  in  a  given 
configuration. 
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